自駕車場景視訊的影像去雨、低光源增強和語義分割

Abstract

近年來，電腦視覺的各領域在單張影像處理的表現優異。然而，當這些架構應用在連續的多幀影像時，常因在架構只有訓練在單張影像，不會藉由前後幀的資訊來彌補缺失，而造成結果有高度的不穩定性和錯誤率。因此需要針對連續的多幀影像設計一個有效的架構。另外，因近年來自駕車的蓬勃發展，電腦視覺在自駕車方面的研究也日益增加。多數任務在正常天氣下有良好的表現，但對於夜晚、下雨和起霧這些極端天氣仍然是極大的挑戰。

本篇論文主要分析三大任務並解決其中的問題：語義分割、去雨和低光源增強，並且針對自駕車的視訊影像去做處理。在每個任務中我們會提出新的架構並加入時序一致性的限制來加強單張影像和連續的多幀影像的結果，並且我們採用了訓練和測試時不同架構的吸想法，讓每個任務都可以達到即時計算的效果，並應用在自駕車系統上。這篇論文主要有兩個貢獻：第一，我們提出的架構成功的解決了在基於單幀影戲那個架構的時序不一致性；第二，我們在各個章節的實驗中驗證了新架構的有效性，大大的在增強視覺上的表現且大大減少各個任務計算所需要的時間。整體來說，這篇論文設計了穩定且有效的架構，在未來不僅針對連續的多幀影像有更廣大的應用，可以更進一步應用在真實自駕車的系統上。

Computer vision has made remarkable progress in single-image processing across various fields in recent years. However, when these architectures are applied to videos, the results are often inconsistent and inaccurate due to a lack of utilization of information from adjacent frames. Therefore, it is essential to design dedicated architectures for video processing. With the growing development of self-driving cars, computer vision research in this area has also increased, with challenging weather conditions such as rain, fog, and night posing significant obstacles.

This thesis focuses on three crucial tasks for processing video frames in self-driving cars: semantic segmentation, rain removal, and low-light enhancement. We propose a novel architecture for each task and introduce temporal consistency constraints to improve the results of both single images and videos. Additionally, we use training and testing with different architectures to achieve real-time processing for each task and apply them to autonomous driving car systems. This thesis has two main contributions: first, our proposed architecture effectively resolves temporal inconsistencies based on the single-frame image architecture; second, we validate the efficacy of the new architecture through a range of experiments in each chapter, significantly improving visual performance and reducing computation time for each task. Overall, this thesis designs a stable and effective architecture with broader applications for videos that can potentially be implemented in real autonomous driving car systems.