ADSP: 先進陰影處理資料集

Abstract

陰影的存在導致了具有較低光強度的區域，削弱了底層資訊並導致高階電腦視覺演算法的失效。因此，作為影像回復的子任務之一，專注於移除陰影並回復背景資訊的陰影去除(shadow removal)是一項重要任務。長久以來，眾多研究者已經提出多種方法。早期方法採用了基於對陰影形成模型的手工設計方法。更近期，隨著大規模資料集的出現，基於深度學習的方法大幅提高了回復品質。然而，作為此類方法的重要基礎，現存的各種資料集具有多項限制。首先，由於資料所包含場景缺乏可見陰影遮擋物，部屬到真實世界的應用時模型可能有所偏差。再者，沒有遮擋物衍伸出受限的相機伏角注定了這些資料集只能有較短的景深和包含俯視圖為主的資料。此外，非陰影區域中嚴重的不一致問題導致了模型能力的不準確評估以及訓練和評估階段之間的偏差。因此，在這篇論文中，一個新的基準，先進陰影處理資料集(Advanced Dataset for Shadow Processing, ADSP)被提出。ADSP採用了新提出基於後處理的合成策略來收集包含可見遮擋物的影像配對。該資料集是第一個包含可見遮擋物的戶外陰影/無陰影影像配對資料集。它為訓練更穩健陰影去除模型提供了很好的監督。同時，不受沒有遮擋物的限制，ADSP在相機角度以及視角種類上有更高的自由度。此外，受益於合成策略的高度可操控性，所提出的ADSP展現了對於不一致問題很好的抑制。統計分析與實驗呈現了ADSP更具挑戰性、更少的遷移、匹配真實世界場景、更好的泛化能力等優勢。最後，作為去除任務上設定一個基準，一個分段細化去除網路(Segmented Refinement Removal Network, SRRN)也被提出。該網路旨在區別兩種去除任務中常見的偽影。更具體的說，採用了一個三階段架構分別進行初步去除、顏色調整以及邊界平滑。在使用去除子網路進行初步去除後，有兩個細化子網，負責處理陰影區域上的顏色偏差，以及半陰影區域上的邊界效應。這兩個子網配有特別設計的損失函數，以專注在回復其目標區域。實驗結果證明了SRRN可以提高視覺愉悅程度，並在提出的ADSP上達到了最先進的陰影去除結果。

The existence of shadow results in low-intensity areas, weakening underlying features and leading to malfunctions in high-level computer vision algorithms. Thus, shadow removal, as a subtask of image restoration, which aims to remove the shadow from the image and recover the background information, is an essential and popular task. For a long time, researchers have proposed different methods to deal with it. Early works adopted hand-crafted methods based on observing shadow formulation models. More recently, with the appearance of large-scale benchmarks, deep-learning-based solutions have further advanced performance in restoration quality. However, as the important base for learning-based methods, the existing benchmarks have several limitations. The lack of visible occluders within the scenes makes them biased when deployed on real-world applications. Restricted camera depression angle doom that they have a short depth of field and are most top-viewed. Severe inconsistency problems in the non-shadow region result in inaccurate estimation of model ability and deviation between training and evaluation. Therefore, in this thesis, a novel benchmark of the Advanced Dataset for Shadow Processing (ADSP) was introduced. The ADSP was built based on a novel synthesizing strategy, a post-processing-based method for collecting image pairs with visible occluders. The ADSP is the first benchmark containing outdoor shadow/shadow-free image pairs with visible occluders, providing excellent supervision for training shadow removal models with better robustness. Meanwhile, without the limitation of visible occluder, the ADSP has a higher degree of freedom regarding camera angle and type of view. Furthermore, benefiting from the high controllability of the synthesizing strategy, the proposed ADSP shows great suppression of inconsistency problems. Statistical analysis and experiments demonstrate that the ADSP has the advantages of being more challenging, less domain shifting, matching real-world scenarios, and having sufficient generalizing capability. Last, a Segmented Refinement Removal Network (SRRN) was proposed as a reference for the removal task. The SRRN was designed to differentiate two artifacts commonly appearing in removal. More specifically, a three-stage structure was adopted to perform preliminary removal, color adjustment, and boundary smoothing, respectively. After the preliminary removal by the shadow removal subnet, two refinement subnets are responsible for addressing the color bias on the shadow region and boundary effect on the penumbra region. These subnets have specially designed loss functions to focus on recovering the target region. Experimental results verify that the SRRN can improve visual pleasantness and achieve state-of-the-art ADSP removal.