由視訊中移除剛體遮蔽物

Abstract

當我們攝影時常常遭遇到的一個問題：當欲拍攝的場景之前被某些遮蔽物所擋住，常見的例如鐵網、欄杆、樹木或圍籬，因為不小心或不得已而將這些遮蔽物攝入影片，透過本研究提出一個有效的方法將這些遮蔽物從影片中移除掉。現存的視訊復原技術大多要求使用者手動勾勒出欲移除物體之粗略輪廓，然而，視訊中大量的影格對於使用者而言是相當沉重而且費時的負擔，尤其是當攝影機是移動的時候。況且，在許多情況下，欲移除物體的外形並非封閉而且可能分佈於整個場景，導致於要在視訊中手動指出欲移除的遮蔽物幾乎是不可行的。 為了解決這個問題，我們延伸基於min-cut的單一影像切割技術至視訊切割來有效地擷取欲移除物體，同時我們利用多種線索來提高視訊切割的可靠性。一旦移除掉遮蔽物，一個基於移動補償的方法被運用來復原被遮蔽的背景，我們透過連續影格之間的對應關係來保持視訊的時間一致性。我們的系統允許輸入的視訊基於無突然搖晃的前提下，任意地移動與旋轉，因此，手提式的攝影機可用以取得輸入影片而毋需三角架或攝影台車的輔助，如此可大大地擴增本系統的應用性。

In this thesis, we propose a technique to reconstruct a “clean” static background scene due to severe occlusions from a video. Most of the current video completion methods require the users to manually draw the rough outlines of the unwanted objects before restoring a video. However, the frame-by-frame task may be very tedious and time-consuming for the users, especially when the camera or objects are moving. In many cases, the occluding objects are not closed forms and dispersed all over the image, e.g., the occluding wire meshes nets, pillars, and fences, with the result that to hint the occluding objects manually is infeasible. To efficiently extract the unwanted foreground objects, we extend the conventional min-cut based image segmentation method to video domain, where multiple cues are integrated to improve the reliability of our system. Subsequently, a motion compensation based video completion method is applied to recover the occluded background scene, where temporal coherence is preserved via the correspondence between consecutive frames. In our system, the camera is allowed to arbitrarily translate and rotate without abrupt shake, namely, a handheld camera can be employed to obtain the input videos. This broadens the applications of our system.