可置換之語音驅動唇形合成方法

Abstract

以影像為基礎的人臉動畫技術已經達到很高的真實度，它可被應用在低頻寬的視訊會議或是在語言學習上扮演虛擬教師角色。可是以影像為基礎的人臉動畫技術需要先為特定使用者拍攝一段五到十分鐘的訓練影片，並加以分析來建立模型以產生動畫，這樣會限制它的應用。我們提出了一個簡單的方法，新使用者只需拍攝幾張特定的影像並利用原先使用者所建立的模型，就能產生出新的人臉動畫。

Image-based videorealistic speech animation achieves significant visual realism such that it can be potentially used for creating virtual teachers in language learning, digital characters in movies, or even user’s representatives in video conferencing under very low bit-rate. However, it comes at the cost of the collection of a large video corpus from the specific person to be animated. This requirement hinders its use in broad applications, since a large video corpus for a specific person under a controlled recording setup may not be easily obtained. Hence, we adopt a simply method which allows us to transfer original animation model to a novel person only with a few different lip images.