基於生成對抗網路之極低光源影像品質增強

Abstract

時下照相機在低光源拍照得到的照片,常常被很強的雜訊影響圖片的品 質,因此低光源影像品質的增進是非常重要且極需解決的問題。另一方面, 近幾年深度學習的方法在各個領域取得許多成功,但是在深度學習模型中的 大量參數,仰賴高品質且大量的數據集來訓練及調整,收集如此的數據集則 需要花費很多時間。在低光源影像品質增強這個任務上,一般的作法需要數 據以成對的形式存在,並且需要在同一個場景同時拍攝短曝光時間、長曝光 時間各一張照片,以此來做為監督式學習使用。但是,在同一個場景拍攝兩 種曝光時間的照片,會需要限制場景內容、控制場景很長一段時間,這樣的 限制也使的一些場景的選擇變得稀少。另外,現存的深度學習方法皆是以整 個模型取代現有的相機處理程序,必須花費額外的力氣去整合兩者的優點。 因此,我們的方法目標是希望藉由連續拍攝的短曝光照片,以及不同場景的 長曝光照片所形成的數據集下,藉由兩階段的訓練,運用生成對抗網路改善 輸出的品質,來達到低光源影像品質增進的目標。並且,我們的方法設計成 可以融入現有相機處理程序,當做原有處理的的預處理階段,因此也能得以 保有現有相機處理程序的各個參數與彈性。在同樣的設定下,我們的方法比 其他方法可以達到更多的品質增進。

Taking photos under low light environments is always a challenge for current imaging pipelines. Image noise and artifacts corrupt the image. Tak- ing the great success of deep learning into consideration recently, it may be straightforward to train a deep convolutional network to perform enhance- ment on such images to restore the underlying clean image. However, the large number of parameters in deep models may require a large amount of data to train. For the low light image enhancement task, paired data requires a short exposure image and a long exposure image to be taken with perfect alignment, which may not be achievable in every scene, thus limiting the choice of possible scenes to capture paired data and increasing the effort to collect training data. Also, data-driven solutions tend to replace the entire camera pipeline and cannot be easily integrated to existing pipelines. There- fore, we propose to handle the task with our 2-stage pipeline, consisting of an imperfect denoise network, and a bias correction net BC-UNet. Our method only requires noisy bursts of short exposure images and unpaired long expo- sure images, relaxing the effort of collecting training data. Also, our method works in raw domain and is capable of being easily integrated into the ex- isting camera pipeline. Our method achieves comparable improvements to other methods under the same settings.