一個擷取與合成高解析度臉部模型與動作之架構

Abstract

我們提出了一個擷取與合成高解析度臉部模型與動作的架構。主要的理論基礎在從物體的漫射或是鏡射，可以容許不同的視角，利用具有漸層亮度的照明模式來估計物體表面的法向量。我們展示了從鏡射求得的法向量是最符合物體表面幾何的，並且可以利用在增加模型的精準度上。而從漫射求得的法向量則可以用來計算表面散射結果的近似解。基於上述的理論，我們開發了可以擷取高解析度靜態頭部模型與動態臉部表演的系統。 靜態頭部模型掃描系統利用了一個球體形狀的漸層亮度照明裝置、兩台單眼數位相機與一台投影機。漸層亮度照明裝置用來投射漸層照明在欲掃描的人物上，被相機擷取下來的影像即利用上述的理論求得臉部上面的法向量。兩台相機與一台投影機組成了一個立體視覺系統，並用來擷取主角的粗略臉部模型。我們再利用了一個最佳化程式，利用求得的鏡射求得的法向量，將粗略的模型變成高解析度。 為了更進一步分析臉部如果根據表情的不同而形變，我們擴充了原先的靜態頭部模型掃描系統，使之可以高速的捕捉臉部動作。利用高速投影機與數位攝影機，我們可以達到每秒擷取三十個高解析度臉部模型的速度。我們再提出了一個利用多項式位移圖來建構與合成臉部表情的技術。最後我們提出了根據球體小波轉換來實作的一個繪製架構，並用來繪製擷取的頭部結果。

We present a framework that captures and synthesizes high resolution facial geometry and performance. In order to capture highly detailed surface structures, a theory of fast normal recovery using spherical gradient illumination patterns is presented to estimate surface normal maps of an object from either its diffuse or specular reflectance, simultaneously from any viewpoints. We show that the normal map from specular reflectance yields the best record of detailed surface shape, which can be used for geometry enhancement. Moreover, the normal map from the diffuse reflectance is able to produce a good approximation of subsurface scattering. Based on the theory, two systems are developed to capture high resolution facial geometry of a static face or dynamic facial performance. The static face scanning system consists of a spherical illumination device, two single lens reflex (SLR) cameras and a video projector. The spherical illumination device is used to cast spherical gradient patterns onto the subject. The captured spherical gradient images are then turned into surface normals of the subject. The two cameras and one projector are used to build a structured-light-assisted two-view stereo system, which acquires a moderate resolution geometry of the subject. We then use the acquired specular normal map to enhance the initial geometry based on an optimization process. To further analyze how facial geometry deforms during performance, we build another facial performance capture system, which is analogous to the previous face scanning system, but employs two high-speed video cameras and a high-speed projector. The system is able to capture 30 facial geometry measurements per second. A novel method based on polynomial displacement maps is presented to cooperate motion capture with real-time face scans, so that realistic facial deformation can be modeled and synthesized. Finally, we present a real-time relighting algorithm based on spherical wavelets for rendering realistic faces under modern GPU architecture.