利用空間-時間雙邊濾波器移除視訊中的雨

Abstract

中文摘要 近年來戶外監視系統已經非常廣泛地被人們所使用，畫面的品質也越來越受到重視，其中最常遇到影響畫面的因素，首推雨滴。雨是日常生活中常常遇到的天候狀況，也是干擾戶外監視系統畫面較為嚴重的因素之一，破壞原本清晰的畫面並導致後續對監視影像內容的判斷及分析，同時造成監視系統裡頭的自動影像分析次系統的誤判，例如特徵檢測、分割及物件辨識等。由於雨滴是在空間上是任意且快速的移動，因此如何將他們從畫面上移除並將具雜訊的畫面還原回復清晰影像，將是一個極具挑戰性的問題。針對此問題，已有許多理論在文獻中提出，然而它們大多只適用在某些限制條件下，而且許多重要參數都還不能自動調整，使得它們無法在實際的監視系統加以應用。 由於雨滴在畫面中會造成複雜的空間及時間變異訊號，因此本篇論文中，我們把雨滴效應移除以畫面去雜訊方式來處理。我們首先提出使用空間-時間雙邊濾波器(spatial-temporal bilateral filter)以作為雨滴濾波器，同時我們也提供軟硬體共用的方式，並以自我偵測參數的方法來達到具備實際應用上必須能簡易安裝的即插即用(plug and play)功能。實驗結果顯示本演算法在効能上超過其它演算法，同時本論文也詳細描述如何以最精簡及有效率的硬體架構來達到所要求的規格，以及即時實現去除雨干擾的效果。我們把此硬體經由製程技術設計流程完成晶片，雛形晶片係利用台積電90nm技術製造，面積為2.352 x 2.354mm²，可在運算速度為125 MHz的情況下，達到即時處理每秒30張1920 x 1080像素的視訊資料能力。

Abstract As outdoor surveillance systems have been widely deployed everywhere, the emphasis on the quality of their frames also rises simultaneously. Rain is a weather phenomenon we frequently encounter in our daily lives, and it is also one the most severe interferences to the frames captured in outdoor surveillance systems, which usually leads to unclear frames and makes users hard to judge and analyze the image information. It also causes the failures of several automatic video analysis subsystems in video surveillance, such as feature detection, segmentation, and object recognition. Due to the random and swift movement of raindrops, it becomes a challenge to remove them from the images and to restore the noisy images back to the clear one. Several previous works have been proposed in literatures; however, most of them can only deal with limited conditions, and several important parameters are required to be set manually, which makes these approaches not feasible for surveillance systems. In this thesis, since the rain drops result in complex space-and-time-varying signals in the images, we treat the rain removal problem as am image de-noising problem. We first propose to employ spatial-temporal bilateral filter as a raindrop filter. Moreover, we also propose software and hardware cooperation methods along with the function of adaptive parameters to achieve the “plug and play” capability required for easy deployment in real applications. The experimental results show that the efficiency of this algorithm surpasses those of other existing algorithms. Detailed hardware architecture analysis is also presented in this thesis to derive an efficient hardware architecture for the proposed algorithm. We also implement the hardware as a chip by using the ASIC design flow with TSMC 90nm technology, and the die size is 2.352 x 2.354mm². Under the working frequency of 125 MHz, the chip can support real-time 30 1920x1080 fps video processing capability.