請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/64560
標題: | 深度先驗模型應用於非盲影像解模糊過程 Deep Prior for Non-Blind Image Deblurring |
作者: | Hung-Chih Ko 葛竑志 |
指導教授: | 丁建均(Jian-Jiun Ding) |
關鍵字: | 非盲解模糊,最大後驗機率,卷積神經網路,深度學習, Non-Blind Deblurring,Maximum a Posteriori,Convolution Neural Network,Deep Learning, |
出版年 : | 2020 |
學位: | 碩士 |
摘要: | 非盲解模糊特指在模糊核已知的前提下對模糊影像進行還原的過程,由於鏡組以及相機運動模式的可計算性,過去經常被使用於太空及航空影像的校正問題中,近年則因手持相機裝置的盛行而使解模糊成為相當重要的技術。直觀方法上可使用逆濾波器對模糊影像做解卷積,然由於模糊核有限的頻寬將導致逆濾波器的高頻區域產生相當大的頻率響應,易導致影像之白雜訊成分被放大,使還原效果受到限制。韋納濾波器(Weiner filter)則可根據影像訊雜比適應性地調整逆濾波器的強度,在當時被認為是最有效率且能有效抑制雜訊的方法。
由於解模糊屬於一類不適定(ill-posed)問題,近年來相關研究中已逐漸提高對原始訊號之先驗模型的重視,可以添加規範項的於優化問題的形式,如l_2、l_1或l_0範數項來規範影像的梯度分佈。Krishnan等人則發現自然影像中的梯度分佈與超拉普拉斯模型(hyper-Laplacian)相似,並發展了快速演算法解決該特定範數所導致的非凸問題,在當時蔚為解模糊之主流。 近年在平行化計算技術的成熟下,更有許多研究以深度學習架構套用至解模糊問題,無論於運算時間或還原效果上皆獲得了相當顯著的改善。然而許多基於深度學習的研究往往倚賴龐大的訓練資料集與運算資源,以致存在實用性的疑慮。並且以端到端(end-to-end)模式訓練的模型容易失去可解釋性,亦或是偏離問題本質上的物理意義,以致存在模型泛化性的疑慮。 本論文旨在提升影像非盲解模糊之準確性,相較許多方法以較為龐大的機器學習模型取得良好表現,本研究則結合傳統規範優化問題中所常使用的半二次方分裂架構,結合輕量化的深度學習模型以改善既有方法所存在的問題。在半二次方分裂架構下,優化問題可分為兩個子問題,即(1)解卷積與(2)先驗規範下的優化問題。前者可經由快速傅立葉算法獲得封閉解,後者的優化方式則往往取決於先驗模型的設計,在此架構下可視為深度學習模型的優化問題。雖此方法獲得具競爭力的表現,但也相當程度地取決於參數選擇,且容易獲得過度模糊化的還原結果。本研究則提出適應解卷積模組(Adaptive Deconvolution Module, ADM)與無參照式影像品質評估(Non-Reference Image Quality Assessment, NR-IQA)以改善此類問題,並提出展開式解卷積網路(Unrolled Deconvolution Network, UDN)以提供較大的學習容積於各個先驗規範模型中。在後續的實驗結果則發現,相較於許多既有的深度學習方法,本研究所提出的方法可使解的收斂速度加快,並達成具競爭力的影像還原表現。 Non-blind deblurring refers to the image restoration process with a knowledge of blur kernels. Due to the calculability of camera lens and motion, it has been applied to the field of astronomical and aerial images at first and become more important with the growth of handheld camera devices. A simplest and intuitive way to handle this problem is by deconvolving with a direct inverse filter. However, it is well-known that the zero entries, which usually lies in high frequency regions, will greatly amplify the white noise that featured with an infinite power spectrum. This problem is resolved by Weiner filter which adaptively adjusts the inverse filter according to the signal-to-noise ratio, such that it has been regarded as an efficient method to suppress the noise. Image deblurring is essentially an ill-posed problem. In recent studies, people have put emphasis on the design of image priors, such as l_2, l_1 or l_0 in the gradient domain, and optimize the original problem with regularization. Krishnan et al proposed a hyper Laplacian prior that is best compiled to the gradient distribution of natural images. Following this observation, a fast deconvolution algorithm was proposed in the same work to handle the non-convexity from this prior. Finally, it became a mainstream in the field of image deconvolution. Recently, with the mature of parallel computing and compactivity of machine learning theories, many works are proposed to elaborate the powerful deep learning model into the deconvolution problem. Although the deep learning based methods takes the advantages on inference speed and accuracy, the training cost on data collection and computing are usually too high to be practical. Besides, when trained in an end-to-end manner, the model is short of explainability and further deviates from the correct physical meaning, such that it may loss the generality. In this thesis, instead of establishing a deep learning model with overwhelmed parameters to enhance deconvolution performance, we rather incorporate light-weight deep learning model into an half quadratic splitting framework which is widely used to solve regulated inverse problems. Under this framework, the overall optimization can be separated into two subproblems: (1) deconvolution and (2) prior-regulated optimization. The former exists a closed-form solution inferred by fast Fourier transform, and the latter usually depends on the prior formulation which can be regarded as an optimization problem of a deep learning model. Although it have shown competitive results, we found the selection of parameters are decisive and the resulting over-smoothed restorations. We proposed an adaptive deconvolution module (ADM) and non-reference image quality assessment (NR-IQA) to improve the performance. In addition, an unrolled deconvolution network (UDN) is proposed to provide a larger capacity for each prior-regulated learning module, and achieve a better convergence speed and performance that are competitive with state-of-the-arts. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/64560 |
DOI: | 10.6342/NTU202000631 |
全文授權: | 有償授權 |
顯示於系所單位: | 電信工程學研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-109-1.pdf 目前未授權公開取用 | 17.01 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。