自我驗證像素內插框架及其相關演算法與硬體架構設計

Abstract

本篇論文中，我們提出了一套自我驗證架構來處理不同的內插問題，包含影像感應測器的彩色濾光片內插，電視訊號的去交錯，影像的超解析放大。這些內差的問題在生活中每天都會遇到，而這一類的問題基於取樣的不足，註定無法還原成完美的高解析度影像。學者提出了許多不同的假設並利用這些假設來內插，問題是這些假設如果不成立，那最後輸出的影像品質就會變得無法預測。我們提出的架構是利用重複內插的收斂性來選出最佳的內插解，架構本身不提供輸出值的預測，而是在幾個候選演算法中，檢查其中一種方法重複內插後的收斂性來判斷內插是否正確。利用此方法，我們可以把不具收斂性的內插法排除，避免產生錯誤的結果.根據我們的客觀與主觀實驗結果，利用自我驗證內插架構比候選演算法提供更好的結果。且品質差距是顯著的。 利用此架構，我們也設計了一套電視用的超解析引擎，能夠將低解析度的電視訊號放大成高解析度的影像。影像訊號中可分結構與材質，我們利用先前提出的自我驗證架構來重建影像的結構並避免錯誤的結果，再利用電腦圖學中的材質合成來生成材質的細紋，給觀看者得到高解析度的感受。我們也分析了對應的硬體架構，利用區塊內遞迴的架構可以大幅降低所需要的頻寬與記憶體用量。 在論文的最後一部分，我們利用人類視覺特性，設計實驗找出人眼對於平面顯示器的動態模糊的極限，並利用此極限設計出移動估計補償畫面插補法來提高高更新率的影片，並有效降低人眼感受到的動態模糊。 綜觀整篇論文，我們的核心觀念是利用人眼視覺特性找出影像處理上的原則與排除冗餘的運算。人類對視覺的認知還只是在粗淺的表面，還有很多未知的領域有待學者分析，在影像處理上也還有非常多的未知數。未來的研究者必能將成果繼續往前推進。

Pixel interpolation is well known as an ill-posed problem of video restoration. In this thesis, a self-validation framework is proposed to solve different kinds of interpolation problems. And corresponding hardware architecture were analysed. In the proposed self-validation framework, multiple algorithms under differ- ent assumptions were performed to generate multiple candidate results. After that, the final estimation of a missing pixel sample was decided adaptively by evaluating the local consistency of each algorithm with a process called double interpolation. By combining the results of different algorithms, the color artifacts are reduced. The proposed framework is applied to three interpolation problems include CFA demosaick, image up-scaling,and de-interlacing. Experimental re- sults demonstrate that the proposed framework improves image and video quality in both subjective and objective assessments. We also implemented a spatial up-sampling hardware for TV scaler using the proposed framework. The corresponding hardware architecture design is also analysed. The proposed tile-based approach can reduce most of bandwidth and make the design practical on low cost hardware. As a results, a tile-based low cost super-resolution hardware is implemented on FPGA. For spatial-temporal interpolation, a real-time hardware-based perception-aware motion-compensated frame interpolation algorithm is proposed. We conducted an experiment to find out the limitations of motion blur perception of human visual system. And these perceptual limits is used to reduce the computational cost of frame interpolation without affecting visual quality. The experimental results show that the proposed low-cost algorithm maintains the visual quality of the interpolation results. Finally, to optimize the trade-off between memory bandwidth and hardware cost, a dedicated hardware architecture design was also proposed. The major contributions of this thesis are: First, to apply human visual system knowledge to image processing and provide high visual quality results without heavy computation. Second, to propose a unified framework for pixel interpolation problems and provide solid simulation results. Finally, to optimize the tradeoff between picture quality and hardware cost to derive a compromising solution for real applications.