多重退化影像之影像增強、域適應、物件偵測及其推廣

Abstract

真實世界的影像經常受到惡劣氣候或不同光照環境影響，使得單張影像中包含多種退化因素，舉例來說，低光影像時常伴隨著圖像噪聲；水下影像時常具有色彩偏移及煙霧效果的退化因素；夜雨影像時常包含雨紋、雨滴、低光效果、光暈效果等。然而目前多數的研究一次僅能處理一種影像退化因素。由於近年深度學習相關方法於電腦視覺展現快速進展，一些研究亦開始致力於處理多重退化影像增強。而對於開發深度學習模型，有兩項重要因素需要考量，其一為訓練資料，因真實世界退化影像與其對應乾淨影像難以同時取得，許多影像增強的演算法因此透過合成資料來訓練模型，然而這會導致訓練的模型在合成影像與真實世界影像間產生域偏移的問題。第二個重要因素為模型設計，目前已有許多模型設計的典範被提出，例如捲積神經網路、生成對抗網路、Transformer、圖像轉換。在本篇論文中，我們將對於多重退化影像增強透過案例討論的方式進行分析與討論，並基於不同的模型設計典範，針對各案例提出我們的解決方案，其中將涵蓋精細設計捲積神經網路及多重領域圖像轉換兩種設計典範。我們也將針對合成影像與真實世界影像之間的域偏移問題進行研究，並透過域適應提供解決方案。最終我們將透過夜雨影像的物件偵測及實例分割，來討論不同模型設計典範間的可應用性及實用性。

A real-world image suffer from adverse weather or lighting condition usually contain multiple degradation at once, for example, low-light image are usually accompanied by noise; underwater image usually accompanied by hazy effect; rainy night image usually contain rain streak, raindrop, low-light and halo effect, etc. However, most of existing works only tackle one degradation once. Thanks to recent learning-based method demonstrate significant advancement in computer vision, some of prior works are devoted to the problem of multiple degradation image enhancement. There are two crucial factor for developing a learning-based method. First one is training data, since the real-world distortion and clear image pairs are hard to obtain for image enhancement, many existing image enhancement algorithms use synthesis data for training, which caused the domain gap for synthesis and real-world images. The second one is model designing, many scheme are proposed to design a deep learning model, for example, CNN, GAN, Transformer, and Image-to-Image Translation. In this paper, we conduct an analysis on multiple degradation image enhancement through case study and provide our solution for each case study based on different model designing paradigm, include carefully design a CNN model, multi-domain Image-to-Image Translation, we also study the problem of domain gap caused by real-world and synthesis images by domain adaptation. We finally provide the discussion and practicality study for different model designing paradigm on single nighttime rainy image object detection and instance segmentation.