利用空疏型最佳化完成內容相關性之影像增強

Abstract

近期，圖像增強的技術被廣泛應用在高解析面板上，其中以超解析圖像技術及高動態圖像合成技術最被重視。雖然有相當數量的研究工作對此議題做探討，然而在邊緣保留、紋理細節、人眼視覺感受，仍然具有非常大的挑戰性。在此論文中，我們不單單是有效地解決邊緣、紋理及人眼感受的議題，我們更進一步對計算機視覺和訊號處理領域中的圖像空疏性質之最佳化進行研究。 第一個議題是超解析圖像技術。超解析技術在公元 1984 年提出，最開始是由多張圖像融合成一張圖像，由於多張圖像之間存在位移量，可以用來提升融合後的圖像畫素。在公元 2000 年前後，圖像生成模型被導入在超解析技術的架構中，從此超解析技術跟去模糊演算法及最佳化求解就息息相關了。不同於傳統的超解析技術是利用多張圖及消去單一模糊核心，我們提出具有空間變動性的單張圖像超解析技術，可以針對單一圖像估測多個模糊核心去完成超解析圖像的重建。更進一步，我們將空間變動性結合人眼視覺感受，使得圖像進行放大的同時，能保持觀測者的視覺感受，在超解析技術的研究進程上，算是一個突破。同時，我們也對圖像空疏性的最佳化求解，提出了一個整合性的架構。 第二個議題是高動態圖像模擬合成技術。高動態圖像合成技術在公元 1996 年提出，利用不同的曝光量，捕捉不同動態區間的圖像，再合成為一張高動態的圖像。同一時期，美國航太總署利用圖像濾波器對單張圖像進行圖像分離，再進而調整整體亮度以及細節對比，模擬出一張相似高動態合成的圖像。由於使用多張圖像融合技術，會帶來模糊、鬼影以及非自然等圖像瑕疵；於是，我們推演美國航太總署的模擬方法，將圖像分離後再增強亮度跟對比的方法。我們避免一般圖像分離會造成的亮度反轉問題，並且我們也導入空疏性、空間變動性及人眼感受等看法。 由高動態圖像合成技術，我們延伸到第三個議題。第三個議題是利用圖像分離技術進行圖像增強，讓圖像在螢幕面板為低背光源的省電模式下，內容依然清晰可視。我們推導了背光源跟圖像的光學關係式，並使用了空疏性質進行圖像分離。此外，我們更進一步提出了一個強化式參數學習的方法，讓系統能夠自主決定使用的圖像分離及圖像增強之參數。 對未來的研究期許，除了讓演算法能在學術領域中謀求突破，我們也希望將所提出的演算法逐一在物聯網及嵌入式系統上實現，用以增加國家在電子、光電及數位內容的科技競爭力；透過軟硬整合、訊號處理及人工智慧，做出產品與服務，對國家的產業經濟，有所裨益。

In recent years, image enhancement technologies have been widely applied to high-resolution displays. Among these image enhancement techniques, super-resolution and HDR-like synthesis are the most important topics. Although a lot of research works are developed to accomplish these two topics, it remains tough to process the regions around edges and the regions full with texture. It is also hard to engage these two topics with human visual perception. In this dissertation, we not merely efficiently deal with the edges, texture regions, and human visual perception. We further consider the optimization methods related to image sparsity which is the current mainstream in the fields of computer vision and signal processing. The first topic is super-resolution. Super-resolution was proposed in Year 1984. The original idea is to combine the information from several input images, register these images in the subpixel level, and fuse together to generate a high-resolution output. Around Year 2000, researchers brought the image formation model into the framework of super-resolution. From then on, super-resolution is highly integrated with image deblurring methods and optimization schemes. Different to the conventional super-resolution methods which deal with multiple images and single blur kernel, we propose a spatially-varying super-resolution using only a single input image. That is, we reconstruct a high-resolution image from only one low resolution image while we deal with multiple blur kernels. Moreover, we extend our spatially-varying super-resolution to a viewing-distance-aware scheme. When we en-large an image, we can maintain the viewer’s visual perceptual constancy. To our knowledge, we are the very first to propose the perceptual super-resolution scheme for this topic. Meanwhile, we also propose a integrated framework for image reconstruction by solving the optimization problem with image sparsity. The second topic is high-dynamic-range-like (HDR-like) image synthesis. HDR was proposed in Year 1996. HDR imaging adopts different dynamic-range input images with different exposures to render a high-dynamic-range image. However, fusing several images to have a HDR image may suffer blurred degradation, ghosting and unnatural artifacts. Meanwhile, in Year 1996, the researchers at NASA adopts a set of filters to decompose the input. Then, they adjust the global base layer and boost the detail layer to synthesize a HDR-like output. To avoid the drawbacks and artifacts in multiple-images scheme, we adopt the image decomposition scheme from a single input to enhance both base layer and detail layer so as to render a HDR-like output. Our proposed HDR-like syntheses cannot only avoid degradation but also consider the image sparsity and human visual system. We can extend the HDR-like synthesis to our third topic. The third topic is image enhancement via sparsity-based decomposition for low-backlighted displays. We deduce the optical relationship between the backlight scale and the input image. We try to approximate the enhanced image on low-backlighted display to the input image on full-backlighted display so as to save the electrical power of the whole system. Here, we adopt the content-aware image decomposition using sparsity-based optimization. Moreover, we propose our reinforcement parameter learning to enable the system to learn the parameters with intelligence. In the future, we will implement the aforementioned algorithms on IoT embedded systems. We not merely pursuit the break-through in the academic research fields of vision, multimedia, signal processing and even artificial intelligence. We also hope that we can further improve the national competitive power in the electronics, display, and digital content industries. Moreover, we wish that our ‘AI+IoT’ embedded integrations and services can also improve the national economic.