請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/55128
標題: | 以三維門卷積與時序性補丁對抗式生成網路之任意形狀影片修復 Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN |
作者: | Ya-Liang Chang 張雅量 |
指導教授: | 徐宏民(Winston Hsu) |
關鍵字: | 影片修復,深度學習,電腦視覺,對抗式生成網路, Video Inpainting,Deep Learning,Computer Vision,GAN, |
出版年 : | 2020 |
學位: | 碩士 |
摘要: | 任意形狀的影片修復是一項很有挑戰性的任務,可以廣泛用於影片編輯(如字幕移除、物體移除)。現有基於補丁修補的方法無法處理非重複性結構(如人臉),而將圖像修復模型直接應用於影片則會導致影片前後不一致。本篇論文提出一種基於深度學習的任意形狀影片修復模型,以三維門控卷積來應對任意形狀遮罩的不確定性,並加上時序性補丁對抗式生成網路(Temporal PatchGAN)來增強影片的一致性。此外,我們收集影片並設計一種任意形狀遮罩生成的演算法,以建立任意形狀影片修復(FVI)資料集,用於訓練和評估影片修復模型。在FaceForensics和FVI資料集上進行的實驗結果顯示,我們的方法優於現有的方法。相關程式碼、結果影片和FVI資料集都在Github上開源 https://github.com/amjltc295/Free-Form-Video-Inpainting。 Free-form video inpainting is a very challenging task that could be widely used for video editing such as text removal. Existing patch-based methods could not handle non-repetitive structures such as faces, while directly applying image-based inpainting models to videos will result in temporal inconsistency (see videos http://bit.ly/2Fu1n6b). In this paper, we introduce a deep learning based free-form video inpainting model, with proposed 3D gated convolutions to tackle the uncertainty of free-form masks and a novel Temporal PatchGAN loss to enhance temporal consistency. In addition, we collect videos and design a free-form mask generation algorithm to build the free-form video inpainting (FVI) dataset for training and evaluation of video inpainting models. We demonstrate the benefits of these components and experiments on both the FaceForensics and our FVI dataset suggest that our method is superior to existing ones. Related source code, full-resolution result videos and the FVI dataset could be found on Github https://github.com/amjltc295/Free-Form-Video-Inpainting |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/55128 |
DOI: | 10.6342/NTU202002144 |
全文授權: | 有償授權 |
顯示於系所單位: | 資訊網路與多媒體研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
U0001-3107202008502400.pdf 目前未授權公開取用 | 1.8 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。