使用生成對抗式網路模擬五官置換

Abstract

本論文的目標是讓使用者模擬將自己的五官之一整形成理想人物的五官，並且使得被置換後的五官可以與使用者其餘未整形之五官恰當地融合。在先前的研究當中，五官置換(face features replacement)的方法通常為先進行五官特徵的偵測(feature detection)接以取代置換並輔以阿爾法混合(alpha blending)為主。然而，當使用者的照片中頭部姿勢與理想人物照片中的頭部姿勢有相當程度的不同之時或是光照情況差異較大之時，即便使用良好的混成方法(blending techniques)，其合成的結果照片也往往不令人滿意。因此在過往五官的整形置換模擬必須限制在使用正臉的照片。本論文採用生成對抗式網路(generative adversarial network, GAN)的架構，並在損失函數(loss function)中加入重建損失(reconstruction loss)以及引導損失(guiding loss)，以得到我們的結果。

Our goal is to replace an individual's facial features with corresponding features of another individual and then fuse the replaced features with the original face. In previous studies, face features replacement can be done by face feature detection and simple replacement. However, when the pose of two faces are quite different, the synthesized image become barely plausible even with good blending techniques. Therefore, current face feature replacement techniques are limited to frontal face only. Our approach leverages the GAN to handle this limitation. Our proposed framework is automatic and does not need any markers on input image. Furthermore, by the introduction of reconstruction loss and guiding loss in GAN, the output image of our approach can preserve the content in source image.