俱多模式感知及適應性應對人類感受之智慧型服務機器人

Abstract

隨著機器人技術的迅速發展，有關於機器人系統與人類日常生活環境深度結合已經成為近年來大家關注的重點之一。為了實現這一目標，機器人系統必須考慮到人類的感受和情緒等資訊，才能表現得更像人類。而這也是人與人之間的交流中最重要的部分之一。 理解人類情感尤其對於服務型機器人至關重要，因為他們需要根據不同情況提供更合適的服務，並表現得更像人類行為，提共更貼心的服務。本文提出了一種用於智能服務機器人感知人類感受的新型多模式方法。而本文中所提及的人類感受，有別於一般的人類情緒，是對於機器人來說更為有用的資訊。 在本文中，我們考慮了類似於機場及養生村等大型公共場所內的移動服務機器人的情況，許多具有各種心理狀態、情緒和感受的人需要不同的幫助。尤其是在養生村這類需要大量看護機器人的地方，能夠理解人類情感對服務機器人非常重要卻也是一大挑戰，如何在適當的情況下提供適當的幫助尤為關鍵。如果能夠透過感知人類的心理狀態而主動提供服務那就更理想。我們通過結合面部表情，語音和用戶動作的情緒識別結果來估計用戶的心理狀態。當中，我們運用了數種最先進的深度學習模型，例如二維和三維的卷積神經網路、遞歸神經網路以及輻射基底類神經網路等。 實驗的結果也顯示，即使在信息有限的情況下，例如其中一項偵測情緒的資訊無法辨識，這樣的方法也可以頗準確地預測人類的心理狀態，若機器人能夠根據人類不同的心理狀態而提供相應的服務，絕對可以促進機器人和人類的交流。 關鍵字：人機互動、深度學習、多模型感測、機器視覺。

With the rapid developments of robotics, the idea of a deep integration of robotic systems in human environments has been raised. To achieve this prospect, the robot systems have to take into account human affects, which is one of the most important aspect of human-human interaction (HHI). Understanding human affects is crucial especially for service robots, so that they can provide more appropriate services base on different situations and most importantly, the robots can act more like human. Rather than emotion, human affects may be a more useful information as it directly refers to how people actually feel. This paper presents a novel multi-modality approach for Intelligent Service Robot to sense human’s feelings. In this thesis, we consider the situation of a mobile service robot inside a large public area such as airport or co-housing district, where many people with various affects require different kinds of help, awareness of user’s affects is especially vital for service robots to provide appropriate assistance to people. We estimate users' affects by combining the results of emotion recognition from facial expression, speech and the motion of users. We use state-of-the-art deep learning models such as 2D and 3D Convolutional Neural Network, Recurrent Neural Network and Radial Basis Function Neural Network. Experiments show that this approach can accurately predict human feelings even in cases with limited information and this can ease the interaction of Robot and Human. Our experiment results also outperform some recent research in audio-visual emotion recognition in terms of accuracy. Besides, this multi-modality approach allows the robot to work even if one of the sources of detection is missed, which is more applicable in real life situation.