將光流法應用於重建式多張影像超解析之可靠方法

Abstract

從一連串觀測到的低解析度影像，合成一張高解析度影像的演算 法，稱之為多張影像超解析度。而重建式多張影像超解析度演算法大 致上可分為兩個步驟: 低解析影像之間的對齊與高解析影像的重建。 在本篇論文中，基於不同張低解析度影像間光強度一致性的假設， 嘗試多種光流法來對齊影像。並且基於來回光流的一致性假設，計算 光流的可信度，將其帶入高解析度影像的重建中，以減少對齊誤差對 重建結果的影響。然而在”分辨力測式卡”這樣的測試資料中，會因 為相機在拍攝過於高頻的圖樣時所產生的錯誤成像，違背光強度一致 性的假設，進而導致光流法對齊失敗。所以我們提出在使用光流法對 齊前，將圖片先進行模糊處理，使得光流法不受此種錯誤成像的影響。 另外，由於現今照片的解析度越來越高，重建高解析度影像需要龐 大的記憶體與時間。本篇論文提出將重建分成多個可平行處理的資料 塊，以減少記憶體用量。並且在硬體方面嘗試使用多執行緒與圖型處 理器加速。重建演算法方面則是提出使用最近鄰居重建法與線性重建 法的結合，進而達到加速的效果。 透過本篇論文提出的方法，能使將光流法運用於多張重建式超解析 度之方法更為可靠。並減少重建的時間與記憶體使用量。

Method of integrating a high-resolution (HR) image from multiple observed low-resolution (LR) images is called multi-frame super-resolution (SR). There are basically two stages of reconstruction-based SR: registration of LR images and reconstruction of HR image. In this thesis, we based on the assumption of intensity consistency, and tried several optical flow methods as registration method. Also, based on another assumption: ”forward-backward flow consistency”, we calculated the confidence of a flow, then brought confidence into HR image reconstruction to reduce the error caused by mis-registration. But in the test sets like ”resolution chart”, there will be some errors caused by some patterns with frequencies that is too high. The errors violates the assumption of intensity consistency, which will cause fail registration of optical flow method. Thus, we proposed to applying blur before calculating the flow. The method can prevent optical flow from failing. Also, due to the resolution of images nowadays becomes higher and higher, which will make the reconstruction of HR image need enormous amount of memory usage and time. The thesis proposed to divide the reconstruction to multiple parallelable data blocks to reduce memory and time usage, and proposed multi-thread and GPU speed-ups. As for algorithm speed-up, we proposed combining nearest neighbors (NN) reconstruction and linear reconstruction to achieve acceleration. With the method proposed by this thesis, we can make using optical flow in multi-frame reconstruction-based SR more robust, and reduce the reconstruction time and peak memory usage.