請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/64560
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 丁建均(Jian-Jiun Ding) | |
dc.contributor.author | Hung-Chih Ko | en |
dc.contributor.author | 葛竑志 | zh_TW |
dc.date.accessioned | 2021-06-16T17:54:35Z | - |
dc.date.available | 2025-03-03 | |
dc.date.copyright | 2020-03-03 | |
dc.date.issued | 2020 | |
dc.date.submitted | 2020-02-26 | |
dc.identifier.citation | [1] A. K. Boyat and B. K. Joshi, “A review paper: noise models in digital image processing,” Signal and Image Processing: An International Journal, vol. 6, no. 2, 2015.
[2] N. Wiener, “Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications,” MIT Press, vol. 8, 1964. [3] A. Levin, Y. Weiss, F. Durand, and W.T. Freeman, “Understanding and evaluating blind deconvolution algorithms,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1964-1971, 2009. [4] L. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms.” Physica D, vol. 60, pp. 259-268, 1992. [5] Y. Wang, J. Yang, W. Yin, and Y. Zhang, “A new alternating minimization algorithm for total variation image reconstruction,” SIAM Journal on Imaging Sciences, vol. 1, no. 3, pp. 248-272, 2008. [6] D. Krishnan and R. Fergus, 'Fast Image deconvolution using hyper-Laplacian priors,' In Advances in Neural Information Processing Systems, pp. 1033-1041, 2009. [7] W. H. Richardson, “Bayesian-based iterative method of image restoration,” Journal of the Optical Society of America, vol. 62, no. 1,pp. 745-754, 1974. [8] L.B. Lucy, “An iterative technique for the rectification of observed distributions,” The Astronomical Journal, vol. 79, pp. 745-754, 1974. [9] D. A. Fish, A. M. Brinicombe, E. R. Pike and J. G. Walker, “Blind deconvolution by means of the Richardson–Lucy algorithm.” Journal of the Optical Society of America A., vol. 12, no. 1, pp. 58-65, 1995. [10] M. Almeida and L. Almeida, “Blind and semi-blind deblurring of natural images,” IEEE Transactions on Image Processing, vol. 19, no. 1, pp. 36-52, 2010. [11] D. Krishnan, T. Tay and R. Fergus, “Blind deconvolution using a normalized sparsity measure,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 233-240, 2011. [12] Q. Shan, J. Jia and A. Agarwala, “High-quality motion deblurring from a single image,” ACM Transactions on Graphics, vol. 27, no. 3, 2008. [13] L. Xu, S. Zheng and J. Jia, “Unnatural L0 sparse representation for natural image deblurring,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1107-1114, 2013. [14] W. Z. Shao, H. B. Li and M. Elad, “Bi-l0-l2-norm regularization for blind motion deblurring,” Journal of Visual Communication and Image Representation, vol. 33, pp. 42-59, 2015. [15] A. Buades, B. Coll and J. M. Morel, “A review of image denoising algorithms, with a new one,” Multiscale Modeling and Simulation, vol. 4, no. 2, pp. 490-530, 2005. [16] C. J. Schuler, H. Christopher Burger, S. Harmeling and B. Scholkopf, “A machine learning approach for non-blind image deconvolution,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1067-1074, 2013. [17] V. Jain and H. Seung, “Natural image denoising with convolutional networks,” In Advances in Neural Information Processing Systems, vol. 21, pp. 769-776, 2008. [18] A. Levin, R. Fergus, F. Durand and W. Freeman, “Deconvolution using natural image priors,” MIT Computer Science and Artificial Intelligence Laboratory, vol. 26, no. 3, 2007. [19] D. Zoran and Y. Weiss, “From learning models of natural image patches to whole image restoration,” In IEEE International Conference on Computer Vision, pp. 479-486, 2011. [20] U. Schmidt, K. Schelten and S. Roth, “Bayesian deblurring with integrated noise estimation,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625-2632, 2011. [21] K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, “Image restoration by sparse 3d transform-domain collaborative filtering,” In Society of Photo-Optical Instrumentation Engineers, vol. 6812, pp. 6, 2008. [22] A. Danielyan, V. Katkovnik and K. Egiazarian, “Bm3d frames and variational image deblurring,” IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 1715-1728, 2012. [23] H. C. Burger, C. J. Schuler and S. Harmeling, “Image denoising: Can plain neural networks compete with bm3d?” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2392-2399, 2012. [24] L. Xu, J. S. Ren, C. Liu and J. Jia, “Deep convolutional neural network for image deconvolution,” In Advances in Neural Information Processing Systems, pp. 1790-1798, 2014. [25] S. Zhang and E. Salari, “Image denosing using a neural network based non-linear filter in the wavelet domain,” In International Conference on Acoustics, Speech, and Signal Processing, 2005. [26] D. Eigen, D. Krishnan and R. Fergus, “Restoring an image taken through a window covered with dirt or rain,” In IEEE International Conference on Computer Vision, 2013. [27] K. Simonyan, A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” In Neural Information Processing Systems, 2015. [28] J. Zhang, J. Pan, W. S. Lai, R. W. Lau and M. H. Yang, “Learning fully convolutional networks for iterative non-blind deconvolution,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3817-3825, 2017. [29] K. Zhang, W. Zuo, S. Gu and L. Zhang, “Learning deep CNN denoiser prior for image restoration,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3929-3938, 2017. [30] K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142-3155, 2017. [31] R. Chang et. al., “One Network to Solve Them All: Solving Linear Inverse Problems Using Deep Projection Models,” In IEEE International Conference on Computer Vision, pp. 5888-5897, 2017. [32] K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition,” In IEEE International Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016. [33] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift.” In International Conference on Machine Learning, pp. 448-456, 2015. [34] A. Krizhevsky, I. Sutskever and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” In Advances in Neural Information Processing Systems, pp. 1097-1105, 2012. [35] F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,” In International Conference for Learning Representations, 2016. [36] J. Delon and A. Houdard, “Gaussian priors for image denoising,” In Denoising of Photographic Images and Video, pp. 125-149, 2018. [37] W. D. Chang, J. J. Ding, Y. Chen, C. W. Chang and C. C. Chang, “Edge-membership based blurred image reconstruction algorithm,” In Asia Pacific Signal and Information Processing Association Annual Summit and Conference, pp. 1-4, 2012. [38] P. Ye, J. Kumar, L. Kang and D. Doermann, “Unsupervised feature learning framework for no-reference image quality assessment,” In IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1098-1105, 2012. [39] D. Temel, M. Prabhushankar and G. AlRegib, “UNIQUE: Unsupervised image quality estimation,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1414-1418, 2016. [40] K. De and V. Masilamani, “No-reference image sharpness measure using discrete cosine transform statistics and multivariate adaptive regression splines for robotic applications,” Procedia Computer Science, vol. 133, pp. 268-275, 2018. [41] J. Anger, C. de Franchis and G. Facciolo, “Assessing the sharpness of satellite images: Study of the planetscope constellation,” In IEEE International Geoscience and Remote Sensing Symposium, pp. 389-392, 2019. [42] J. Kumar., F. Chen and D. Doermann, “Sharpness estimation for document and scene images,” In IEEE International Conference on Pattern Recognition, pp. 3292-3295, 2012. [43] Z. Wang and G. Yuan, “Image noise level estimation by neural networks,” In International Conference on Materials Engineering and Information Technology Applications, 2015. [44] G. Chen, F. Zhu and P. A. Heng, “An efficient statistical method for image noise level estimation,” In IEEE International Conference on Computer Vision, pp. 477-485, 2015. [45] D. Martin, C. Fowlkes, D. Tal and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” In IEEE International Conference on Computer Vision, vol. 2, pp. 416-423, 2001. [46] D. Kingma and J. Ba. Adam, “A method for stochastic optimization,” In International Conference for Learning Representations, 2015. [47] A. Paszke et al., “PyTorch: an imperative style, high-performance deep learning library,” In Advances in Neural Information Processing Systems, pp. 8024-8035, 2019. [48] L. Sun, S. Cho, J. Wang and J. Hays, “Good image priors for non-blind deconvolution - generic vs. specific,” In European Conference on Computer Vision, pp. 231-246, 2014. [49] J. Kruse, C. Rother and U. Schmidt, “Learning to push the limits of efficient fft-based image deconvolution,” In IEEE International Conference on Computer Vision, pp. 4586-4594, 2017. [50] J. Pan, Z. Lin, Z. Su and M. H. Yang, “Robust kernel estimation with outliers handling for image deblurring,” In IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2800-2808, 2016. [51] L. Sun, S. Cho, J. Wang and J. Hays, “Edge-based blur kernel estimation using patch priors,” In IEEE International Conference on Computational Photography, pp. 1-8, 2013. [52] J. M. Bioucas-Dias and M. A. Figueiredo. “A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Transactions on Image Processing, vol.16, no. 12, pp. 2992-3004, 2007. [53] A. Beck and M. Teboulle. “A fast iterative shrinkage thresholding algorithm for linear inverse problems,” SIAM journal on imaging sciences, vol. 2, no. 1, pp.183-202, 2009. [54] S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein,” Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations and Trends in Machine Learning, vol. 3, no. 1, pp. 1-122, 2011. [55] A. Krizhevsky, I. Sutskever and G. E. Hinton, “Imagenet classification with deep convolutional neural networks.” In Advances in Neural Information Processing Systems, pp. 1097-1105, 2012. [56] R. Zeyde, M. Elad and M. Protter, “On single image scale-up using sparse representations,” In International Conference on Curves and Surfaces, pp. 711-730, 2010. [57] O. Kupyn, T. Martyniuk, J. Wu and Z. Wang, “Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better,” In IEEE International Conference on Computer Vision, pp. 8878-8887, 2019. [58] B. Lu, J. C. Chen and R. Chellappa, “Unsupervised domain-specific deblurring via disentangled representations,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 10225-10234, 2019. [59] D. Gong, Z. Zhang, Q. Shi, A. v. d. Hengel, C. Shen, Y. Zhang, “Learning an optimizer for image deconvolution,” arXiv preprint arXiv:1804.03368, 2018. [60] S. Roth and M. J. Black, “Fields of experts: A framework for learning image priors,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 860-867, 2005. [61] U. Schmidt and S. Roth, “Shrinkage fields for effective image restoration,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2774-2781, 2014. [62] U. Schmidt, C. Rother, S. Nowozin, J. Jancsary and S. Roth, “Discriminative non-blind deblurring,” In IEEE Conference on Computer Vision and Pattern Recognition, pp. 604-611, 2013. [63] M. Gholizadeh-Ansari, J. Alirezaie and P. Babyn, “Deep learning for low-dose CT denoising using perceptual loss and edge detection layer,” Journal of digital imaging, pp. 1-12, 2019. | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/64560 | - |
dc.description.abstract | 非盲解模糊特指在模糊核已知的前提下對模糊影像進行還原的過程,由於鏡組以及相機運動模式的可計算性,過去經常被使用於太空及航空影像的校正問題中,近年則因手持相機裝置的盛行而使解模糊成為相當重要的技術。直觀方法上可使用逆濾波器對模糊影像做解卷積,然由於模糊核有限的頻寬將導致逆濾波器的高頻區域產生相當大的頻率響應,易導致影像之白雜訊成分被放大,使還原效果受到限制。韋納濾波器(Weiner filter)則可根據影像訊雜比適應性地調整逆濾波器的強度,在當時被認為是最有效率且能有效抑制雜訊的方法。
由於解模糊屬於一類不適定(ill-posed)問題,近年來相關研究中已逐漸提高對原始訊號之先驗模型的重視,可以添加規範項的於優化問題的形式,如l_2、l_1或l_0範數項來規範影像的梯度分佈。Krishnan等人則發現自然影像中的梯度分佈與超拉普拉斯模型(hyper-Laplacian)相似,並發展了快速演算法解決該特定範數所導致的非凸問題,在當時蔚為解模糊之主流。 近年在平行化計算技術的成熟下,更有許多研究以深度學習架構套用至解模糊問題,無論於運算時間或還原效果上皆獲得了相當顯著的改善。然而許多基於深度學習的研究往往倚賴龐大的訓練資料集與運算資源,以致存在實用性的疑慮。並且以端到端(end-to-end)模式訓練的模型容易失去可解釋性,亦或是偏離問題本質上的物理意義,以致存在模型泛化性的疑慮。 本論文旨在提升影像非盲解模糊之準確性,相較許多方法以較為龐大的機器學習模型取得良好表現,本研究則結合傳統規範優化問題中所常使用的半二次方分裂架構,結合輕量化的深度學習模型以改善既有方法所存在的問題。在半二次方分裂架構下,優化問題可分為兩個子問題,即(1)解卷積與(2)先驗規範下的優化問題。前者可經由快速傅立葉算法獲得封閉解,後者的優化方式則往往取決於先驗模型的設計,在此架構下可視為深度學習模型的優化問題。雖此方法獲得具競爭力的表現,但也相當程度地取決於參數選擇,且容易獲得過度模糊化的還原結果。本研究則提出適應解卷積模組(Adaptive Deconvolution Module, ADM)與無參照式影像品質評估(Non-Reference Image Quality Assessment, NR-IQA)以改善此類問題,並提出展開式解卷積網路(Unrolled Deconvolution Network, UDN)以提供較大的學習容積於各個先驗規範模型中。在後續的實驗結果則發現,相較於許多既有的深度學習方法,本研究所提出的方法可使解的收斂速度加快,並達成具競爭力的影像還原表現。 | zh_TW |
dc.description.abstract | Non-blind deblurring refers to the image restoration process with a knowledge of blur kernels. Due to the calculability of camera lens and motion, it has been applied to the field of astronomical and aerial images at first and become more important with the growth of handheld camera devices. A simplest and intuitive way to handle this problem is by deconvolving with a direct inverse filter. However, it is well-known that the zero entries, which usually lies in high frequency regions, will greatly amplify the white noise that featured with an infinite power spectrum. This problem is resolved by Weiner filter which adaptively adjusts the inverse filter according to the signal-to-noise ratio, such that it has been regarded as an efficient method to suppress the noise.
Image deblurring is essentially an ill-posed problem. In recent studies, people have put emphasis on the design of image priors, such as l_2, l_1 or l_0 in the gradient domain, and optimize the original problem with regularization. Krishnan et al proposed a hyper Laplacian prior that is best compiled to the gradient distribution of natural images. Following this observation, a fast deconvolution algorithm was proposed in the same work to handle the non-convexity from this prior. Finally, it became a mainstream in the field of image deconvolution. Recently, with the mature of parallel computing and compactivity of machine learning theories, many works are proposed to elaborate the powerful deep learning model into the deconvolution problem. Although the deep learning based methods takes the advantages on inference speed and accuracy, the training cost on data collection and computing are usually too high to be practical. Besides, when trained in an end-to-end manner, the model is short of explainability and further deviates from the correct physical meaning, such that it may loss the generality. In this thesis, instead of establishing a deep learning model with overwhelmed parameters to enhance deconvolution performance, we rather incorporate light-weight deep learning model into an half quadratic splitting framework which is widely used to solve regulated inverse problems. Under this framework, the overall optimization can be separated into two subproblems: (1) deconvolution and (2) prior-regulated optimization. The former exists a closed-form solution inferred by fast Fourier transform, and the latter usually depends on the prior formulation which can be regarded as an optimization problem of a deep learning model. Although it have shown competitive results, we found the selection of parameters are decisive and the resulting over-smoothed restorations. We proposed an adaptive deconvolution module (ADM) and non-reference image quality assessment (NR-IQA) to improve the performance. In addition, an unrolled deconvolution network (UDN) is proposed to provide a larger capacity for each prior-regulated learning module, and achieve a better convergence speed and performance that are competitive with state-of-the-arts. | en |
dc.description.provenance | Made available in DSpace on 2021-06-16T17:54:35Z (GMT). No. of bitstreams: 1 ntu-109-R06942148-1.pdf: 17422068 bytes, checksum: 9daa7e54dc49bcf7802246ac6d9f1c5e (MD5) Previous issue date: 2020 | en |
dc.description.tableofcontents | 誌謝 i
中文摘要 ii ABSTRACT iv CONTENTS vi LIST OF FIGURES viii LIST OF TABLES xi Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Primary Contribution 3 1.3 Organization of the Thesis 4 Chapter 2 Review on Existing Image Deconvolution Algorithms 5 2.1 Image Model 5 2.2 Traditional Method 5 2.2.1 Direct Inverse Filter 5 2.2.2 Weiner Filter 6 2.2.3 Richardson-Lucy Method 8 2.3 Maximum a Posteriori Based Method 10 2.3.1 Tikhonov regularization 12 2.3.2 Total Variation Regularization and Half Quadratic Splitting 12 2.3.3 Hyper-Laplacian Prior 15 2.3.4 Alternative Norm-Based Priors 16 2.4 Summary 19 Chapter 3 Review of Deep Learning Based Deconvolution Methods 21 3.1 From Multi-Layer Perceptron to Convolution Neural Networks 21 3.2 Convolution Neural Network with Small Kernels 23 Chapter 4 Proposed Non-Blind Deconvolution Algorithm 29 4.1 Adaptive Deconvolution Module 29 4.1.1 Adaptation on I Subproblem 29 4.1.2 CNN Denoiser For Z Subproblem 36 4.1.3 No-Reference Image Quality Assessment 37 4.1.4 Summary 41 4.2 Unrolled Deconvolution Network 44 4.2.1 Layers for I Subproblem and Z Subproblem 44 4.2.2 Training Procedures 46 4.2.3 Combination of ADM and UDN 47 Chapter 5 Simulation Results 49 5.1 Database 49 5.2 Convergence Properties 51 5.3 Runtime Comparison 52 5.4 Comparison with State-of-the-Art 53 5.4.1 Visualization 54 5.4.2 Quantification Results 60 Chapter 6 Conclusion and Future Work 62 6.1 Conclusion 62 6.2 Future Work 63 REFERENCE 64 | |
dc.language.iso | en | |
dc.title | 深度先驗模型應用於非盲影像解模糊過程 | zh_TW |
dc.title | Deep Prior for Non-Blind Image Deblurring | en |
dc.type | Thesis | |
dc.date.schoolyear | 108-1 | |
dc.description.degree | 碩士 | |
dc.contributor.oralexamcommittee | 許文良(Wen-Liang Hsue),郭景明(Jing-Ming Guo) | |
dc.subject.keyword | 非盲解模糊,最大後驗機率,卷積神經網路,深度學習, | zh_TW |
dc.subject.keyword | Non-Blind Deblurring,Maximum a Posteriori,Convolution Neural Network,Deep Learning, | en |
dc.relation.page | 70 | |
dc.identifier.doi | 10.6342/NTU202000631 | |
dc.rights.note | 有償授權 | |
dc.date.accepted | 2020-02-27 | |
dc.contributor.author-college | 電機資訊學院 | zh_TW |
dc.contributor.author-dept | 電信工程學研究所 | zh_TW |
顯示於系所單位: | 電信工程學研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-109-1.pdf 目前未授權公開取用 | 17.01 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。