運用情緒和眼神建立人機互動

Abstract

人類的眼睛和情緒是一種強力的非語言溝通工具 : 除了表達一個人的興趣、專注和意圖外，也被用來管理面對面的互動。然而，在人機互動的領域中，鮮少應用實際的整合了眼神和情緒。本論文致力於建立一個能自動偵測、感應、理解人類的眼神和情緒並採取相對應的行動的智慧系統，以增進人機互動的流暢性，使機器人的行為能更像人類。此外，藉由辨識人類情緒，機器人能擁有更好的自主能力適時地對人類提供幫助。 掛載於實驗室開發的輪型機器人上的此系統，在運作時會先擷取目標的臉部影像，利用生成對抗網路對進入的影像做超解析，並使用卷積神經網路進行眼神和情緒的辨識，最後透過漸進式隱馬可夫模型評估目標的意圖並採取相對應的行動。

Human eyes and emotions represent strong non-verbal communication tools. They not only give cues about interest, attention, and the intention of people, but also manage several kinds of social face-to-face interactions. However, only a few applications take into account human gaze and emotions in the field of human-robot interaction (HRI). This thesis aims to create a comprehensive intelligent system to automatically sense, understand, and react to human eye gaze and emotions, in order to both improve HRI smoothness and make robots behave more human-like. Besides, by identifying emotional information, ro-bots might be able to automatically offer help to people in need. The online system, mounted on a mobile robot, detects human faces from 2D images, uses super-resolution algorithms on a generative adversarial network (GAN) to improve the facial input, recognizes the gaze and emotional behavior based on convolutional neu-ral networks (CNN), and uses a novel incremental variant of a hidden Markov model (iCHMM) to estimate the intention of the people interacting with the robot. Finally, with the proposed interaction model, the robot can also act according to its own intentions.