以對抗網路學習高解析度影像之高頻率成份來增強影像之解析度

Abstract

近年來，隨著數位顯示器的進步，過去製作的數位內容已無法與之匹配，因此運用深度學習方法，使影像解析度增強的議題，變得廣為人們所研討。本論文以對抗網路為基礎，針對圖像超解析度(Super Resolution)進行研究，不同於過往直接於空間域(Spatial Domain)對影像進行解析度增強之方式，而是透過傅立葉轉換將影像由空間域轉換至頻率域(Frequency Domain)，將所得之影像頻譜圖乘上權重矩陣，利用深度網路學習影像中的高頻資訊，最終完成對高解析度影像的重建。實驗中將影像分為無字幕影像及帶有字幕之影像，並對其解析度進行三種倍率的增強實驗。據實驗結果顯示，對於無字幕影像進行三種倍率的增強時，與傳統的影像插值法及SRGAN進行比較，本論文所提出之方法在PSNR及SSIM的比較上是優於另外兩種方法。而對於帶有字幕之影像進行解析度增強，將字幕從影像中擷取出來，分別對其進行解析度增強，從實驗結果可知本研究所提出之方法，亦是優於影像插值法及SRGAN。

In recent years, with the development of digital displays, the digital content produced in the past can’t match with it. Therefore, the issue of image resolution enhancement using deep learning methods has become widely discussed. In this paper, we study the super resolution of images based on the adversarial network. Instead of directly enhancing the image resolution in the spatial domain in the past, we convert the image from the spatial domain to the frequency domain by Fourier transform, then multiplies the result image spectrum map by the weight matrix, and use the Deep learning to learn the high-frequency information in the image, consequently the high resolution image reconstruction will be completed. In the experiment, the images were divided into two sections: images without subtitles and images with subtitles, and the resolution was enhanced at three different magnifications. The experimental results showed that the proposed method is superior to the traditional image interpolation method and SRGAN in terms of PSNR and SSIM when compare the three enhancement rates with the other two methods. For the resolution enhancement of images with subtitles, the subtitles are extracted from the images , after that the resolution is enhancement respectively. From the resolution enhancement of images with subtitles as we know, the method proposed in this study is better than the image interpolation method and SRGAN.