高畫質視訊中即時物體切割之演算法與架構設計

Abstract

視訊切割技術已被廣泛用於影像編碼、虛擬通話、影像特效處理及智慧型監控系統中。藉由分析並分離影像中的構造成為受注目前景區域、背景區域及其他組成元素，視訊切割將影像轉換為有意義的遮罩格式。經過後續處理後，遮罩便可以表示影像中的物件。這些結果可以被物件追蹤、行為分析等後續演算法採用來進一步根據物件特性分析影像內容，以實現先進的智慧型自動影像分析系統。相關的應用包括智慧型監控系統、無人居家看護系統、公共場所人身安全警示系統等。 隨著智慧型自動分析技術的演進，先進系統發展目標趨向全自動化以及系統處理範圍、影像解析度、處理速度、運算效率上的提昇。為了減少主要伺服器的運算負擔、有效利用系統頻寬，部分運算需要在攝影機端預先處理完，只將重要的資訊傳輸回伺服器。由於視訊切割將完整的影像分析後僅留下遮罩的資料，在視訊切割之後，攝影機端對伺服器的傳輸量可大幅減少。而即時視訊切割演算法的也有高度的共通性，因此十分適合整合於攝影機端。 在本論文中，我們提出一種利用遮罩資料連續特性的運算方法，加速在視訊物體切割演算法中被廣泛運用的多種運算，以支援系統大處理範圍的高解析度影像即時視訊物體切割。不同於以往依照像素的格式儲存並運算，這個處理方式使用了連續格式，可以有效減少系統傳輸量、並減少多餘的運算。 我們也針對此運算方法設計了硬體架構設計以加速運算來滿足即時視訊切割的需求。這個硬體架構使用了兩段式資料再利用模式，並針對特定運算使用對應的儲存格式，進一步減少暫存資料存取使用的系統頻寬。此設計可以與終端攝影機整合，用於新一代智慧型影像分析系統中。

Video Segmentation has been widely used in video coding, virtual communication, image / video special effect processing, and intelligent surveillance systems. Video segmentation separates a video frame into meaningful binary masks. After post-processing, these masks can represent objects in a video. Then the data can be used by object tracking or behavior analysis algorithms to further analyze the video content according to object characteristics, realizing modern intelligent video analysis systems. Related applications include intelligent video surveillance systems, automatic home caring systems, and public place safety alarm systems. As intelligent video analysis technique evolves, the goal of advanced systems turns to fully-automatic systems which can handle enlarged coverage and resolution with faster frame-per-second (fps) rates. In addition, to alleviate the computation load of the central server of a system and to attain efficient system bandwidth usage, part of the computation should be handled at terminal side, i.e., at camera side. The computed data are then transmitted to central server occupying less bandwidth. Since video segmentation leaves binary masks instead of the entire image frame, putting video segmentation at terminal side greatly reduces the amount of transmission. Furthermore, the high compatibility between video segmentation algorithms ensures implementation integrity. It is acknowledged that video segmentation can be integrated into camera devices. In this thesis, a run-processing method which accelerates multiple functions widely used in video segmentation algorithms is proposed to support real-time high-resolution video object segmentation algorithms. Instead of using conventional pixel-based solutions, the presented method utilizes run-coding to reduce redundant computation and at the same time lower system bandwidth requirements. The proposed method is carefully designed to ensure a practical hardware implementation. The corresponding hardware architecture is introduced and fabricated as a single-chip solution. This architecture possesses a two-phase partial-data reuse scheme. Moreover, corresponding storage format is adopted for certain functions, thus further diminishing overall system bandwidth. The chip implementation can be integrated into camera devices and serves as a terminal-device accelerator in next-generation intelligent video analysis systems.