使用生成對抗式網路進行基於語義之影像增強

Abstract

隨著深度學習技術之發展，現在可使用深度學習模型來取代人工對影像進行調整、美化之過程，進而減少人工修圖所需之人力成本，以及修圖需要大量領域知識之挑戰。過往使用深度學習模型進行影像美化之方法雖已取得很大的成果，然而其大多未考慮到影像的語義資訊，亦即目前不少的方法皆是對整張影像進行同一種風格之美化，然而在一張影像中，我們可能會期望不同語義區域會有不同的調整方向，因此，我們希望能透過將語意分割模型之輸出引入至影像增強模型之中，讓影像增強模型在獲得語意資訊之後，能夠在各個語意區域上做出不同方向的調整與美化，使得整體影像獲得更好的美化效果。除了模型修改之外，我們也針對欲美化之語意收集了資料集，期望模型能透過我們自己所收集的資料集，學習到我們認為各個語意在怎樣的外觀或風格下是好看的。經實驗表明我們所提出的模型與方法確實能達成針對語意進行影像美化之目標。

With the advancement of deep learning techniques, it is now possible to replace the manual process of adjusting and beautifying images with deep learning models. This approach helps reduce the manpower cost associated with manual retouching and addresses the challenges of requiring extensive domain knowledge in image editing. While many deep learning-based methods have achieved significant success in image enhancement, most of them do not consider semantic information in the images. In other words, many existing methods focus on uniformly enhancing the entire image with a single style.

However, in a given image, it is often desirable to have different adjustment directions for various semantic regions. Therefore, by incorporating the output of a semantic segmentation model into the image enhancement model, we aim to enable the image enhancement model to make different adjustments and beautifications in different semantic regions. This allows the model to achieve better overall aesthetic improvements. In addition to modifying the model, we have collected datasets specific to the semantics we aim to beautify. We hope the model can learn from our collected datasets the desirable appearances or styles for different semantics. Experimental results indicate that our proposed model and approach can indeed achieve the goal of semantically guided image enhancement.