基於語意與編碼簿的人臉影像修復

Abstract

本篇論文致力於解決影像重建中的盲人臉修復問題，在真實世界的影像拍攝與應用中，影像常因低解析度、雜訊、模糊和壓縮失真等未知因素而受損，因此在盲人臉修復的任務中，我們希望能夠訓練一個影像修復模型，在只有退化影像當作輸入的情況下來復原出高品質的人臉影像。在這篇論文中，我們提出了一個名為「語意與編碼簿的人臉影像修復」（DPFR）新框架，該框架整合了幾何先驗和生成先驗，以有效地進行盲人臉修復。為了結合前述兩者先驗，我們將人臉語意遮罩當作訓練資料的一部份來訓練離散編碼簿，並且透過語意感知轉換模組(Semantic-Aware Conversion Module, SAC Module)將語意資訊融合到主解碼器中。最終實驗結果顯示，藉由同時採用語意及編碼簿先驗，我們的方法測試在合成以及真實世界的資料集上，相較於既有的方法在量化指標與視覺比較上都有更好的表現。

This thesis addresses the problem of blind face restoration in image reconstruction. In real-world image photography and corresponding applications, images are often degraded due to low resolution, noise, blur, compression artifacts, and other unknown factors. Therefore, the goal of blind face restoration is to train an image restoration model that can recover high-quality facial images using only the degraded images as input. In this thesis, we propose a novel framework named "Dual Prior Face Restoration" (DPFR), which integrates geometric and generative priors to perform blind face restoration effectively. To combine these two types of priors, we incorporate face semantic masks to the inputs to train a discrete codebook and use a Semantic-Aware Conversion (SAC) module to integrate semantic information into the main decoder. The final experimental results demonstrate that by leveraging the advantages of semantic and codebook prior, our method performs competitively against existing methods in both quantitative metrics and visual comparisons on both synthetic and real-world datasets.