以雙攝影機為基礎之視訊形變及其在視訊會議之應用

Abstract

在視訊會議中，錯誤的使用者眼神方向的問題一直存在。使用者總是會覺得對方沒在看自己而是在看其他地方。正確的眼神方向可以讓使用者覺得比較自然，進而更專心在彼此的交談中。這篇論文提出了新的一個虛擬視角合成系統： 使用者只要將兩台網路攝影機安置在電腦螢幕的兩旁（左邊與右邊），系統就會合成出一張眼神轉到正前方而且高品質畫面的虛擬影像。在先前其他相關的研究中， 要即時合成出虛擬的視訊影像需要大量的CPU 計算。如果使用者希望享受一個高品質而且即時生成結果的系統， 往往必須求助於專門的硬體設備。本篇論文所提出的方法是基於軟體執行來實現， 而所提出的方法(scanline based matching algorithm)可以讓執行效率媲美硬體加速。為了讓合成出來的虛擬影像看起來更為自然， 我們提出了一個新演算法(modified dynamic programming)。並且對遮蔽問題作了一些改進。論文最後也附上一些實驗結果與數據。

In recent years, eye contact problem always exists in daily video conferencing. Users will feel more comfortable and pay more attention to the conversation with proper eye contact. This thesis proposes a novel virtual view synthesis system. Given two video streams from two cameras located in either side of the monitor (ex: Left and right), the system synthesizes a high quality view-corrected virtual image. In the previous work, synthesized images take lots of time for CPU computation. Special hardware-support may be the only solution if the application needs to be high quality and real-time. However, the performance of our proposed fast scanline based matching algorithm can reach real-time, comparing to certain conventional matching algorithms. To achieve natural virtual image synthesizing, the modified dynamic programming approach is adopted. Artifacts and occlusion problems are also handled. A number of examples demonstrate the high quality results of the proposed algorithm.