基於雙流擴散模型與物理渲染之去除單張圖像反射現象

Abstract

光線反射是生活中無處不在的自然現象，這使得一般攝影器材在捕捉影像時，難免會拍到一些不希望出現的反射物體，輕則影響視覺上的美感，重則會嚴重影響到下游電腦視覺相關任務的進行，因此如何有效地去除影像中的反射，是此領域中長久被關注的課題。近年來，基於深度學習的方法雖然與傳統方法相比已有很大的進步，然而模型的效能仍會因為以下兩個根本的問題受到限制，分別是過度簡化的反射模型假設以及合成資料與真實反射影像之間的差距。為此我們首先透過引入擴散模型的方法，來減少模型對於假設的依賴，並利用雙流網路的架構來同時預測殘差和反射層，加以增強擴散模型捕捉複雜資料分佈的能力。此外採用物理渲染的技術來生成訓練所需要的資料集，以縮小合成數據與真實世界影像之間的差距。在測試資料上的實驗結果表明，我們的模型只需使用相較於過去方法約10%的合成訓練資料，便能展現出與主流方法相媲美的性能。通過將擴散模型引入此研究領域，我們的工作展示了其在低層次視覺任務中的潛力，為該領域的後續發展跨出了新的一步。

Reflections are ubiquitous in our daily lives, making it inevitable for common photographic equipment to capture unwanted reflected objects when taking images. At best, these reflections affect the visual aesthetics of the image; at worst, they can severely impact the performance of downstream computer vision tasks. As a result, effectively removing reflections from a single image has long been a focus of attention in this field. In recent years, although deep learning-based methods have made significant progress compared to traditional approaches, their performance is still limited by two fundamental issues: overly simplified reflection model assumptions and the domain gap between synthetic and real-world reflection images. We first introduce a diffusion model-based approach to reduce dependency on assumptions, using a dual-stream network architecture to simultaneously predict residuals and reflection layers, thereby enhancing the diffusion model's ability to capture complex data distributions. Additionally, we employ physically based rendering techniques to generate the necessary training datasets, narrowing the gap between real-world images and synthetic data. Experimental results on benchmark data show that our model can achieve performance comparable to state-of-the-art methods using only about 10% of the synthetic training data required by previous approaches. By introducing diffusion models into this research area, our work demonstrates their potential in low-level vision tasks, marking a new step forward for the field's development.