使用三維 N3Net 和細節水平圖作影片去噪

Abstract

由於硬體的限制，雜訊在攝影學上是一個不可避免的問題。為了處理雜訊學術上已經有了許多去雜訊的方法。其中一個稱為影片去雜訊，也就是利用影片中其他幀來幫忙對每個幀去雜訊。本論文中提出一個使用 N3NET 作為骨幹的模型。我們將這個概念延伸到多影像的去雜訊問題。另外我們還訓練另一個子模型來學一個所謂的細節水平圖。細節水平圖之於影像的概念類似於雜訊水平圖之於雜訊。整個模型最後使用細節水平圖和原本的影像共同預測最後的結果。利用 3D 的 N3NET 可以在視覺上得到和前人成果類似的品質。並且使用接近真實的細節水平圖，我們可以得到再進一步更好的結果。

Noise is an inevitable problem of photography due to hardware limitations. To tackle with it, researchers have developed various kinds of denoising methods. One of the methods use neighbor frames from video to help denoising each frames, which is so-called video denoising. In this paper, we use N3NET as backbone, which leverages neighbor patches to help denoising, and extend the concept of it to multiple images denoising problem. Furthermore, we train another sub-model to learn a so-called detail-level map of images, an analogy to noise-level map of noise from photography terms. In the end we use both detail-level map and original frames to predict the denoised result. We show that by using 3D N3Net we can have similar visual quality with state-of-the-art methods. And with close-to-ground-truth detail-level map, we can further improve the result.