以混合互動強化學習方式輔助機器人行動計劃用於孩童情感支持

Abstract

社交輔助機器人(SAR)為新興的多學科研究領域，在教育、健康管理和高齡照護有著應用潛能。傳統上來說，機器人的研究題目多著重如動作、自動導航或電腦視覺等等，相較傳統而言，社交輔助機器人能運用人工智慧和機器人學習工具不只能賦予機器人和人類在社會情境下互動，而且能用於情感支持，特別是在人口中較脆弱的一群，如未成年者和高齡者。 在過去幾年，有些研究致力於探討機器人如何以自然語言處理和輔助機器以融入孩童的人際互動。但尚未有方法提出以情緒理論為行動計劃系統的核心要件。以情感理論為行動計劃制度的核心要件，能使機器人之行動計劃模組選擇最有可能改善小兒科患者情感狀況。 本論文以開發RoBoHoN互動決策計畫模組，來挑戰混合互動強化學習的典範。RoBoHoN為在安卓OS執行的社交型機器人平台，目標為探索機器人是否有參與，並且改善孩童情感狀態之可能性。為了分類病童當下的情感狀態，孩童的臉部表情將被捕捉並且經由處理。接著，運用兒童醫療輔導師的專門知識，配合遠端方式操作，行動計劃模組能夠以互動性的方式學習，在病童的特殊情感狀態下做出最適合的行為與反應。 為了應證所提出研究方法的可用性，我們首先執行健康小學生的試驗研究。我們以應用兒童心理學為基礎的結構型問卷，評估機器人對受試者一般性的情感狀況影響。我們的結果顯示：不只已提出的架構能促使孩童和機器人可靠和流暢的社交互動，加上我們使用的方法，機器人還能增強下列三點能力 • 具有能力改善兒童的情感狀態 • 能夠建立並維持和孩童的社交參與感 • 建立與孩童的友好關係 根據上述研究發現，我們相信此平台具有運用以發展Wizard-of-Oz (遠端操控)於真實環境的潛能(如課程或醫院)。此外，為了改進孩童的情感狀況，機器人將於學習最佳行為後，發行自動反應的版本。

Socially Assistive Robotics (SAR) is an emerging multidisciplinary field of study that has potential applications in education, health management and elder care. Traditionally, research in Robotics focuses on topics like motion, navigation or robot vision. A SAR on the other hand, may leverage tools from the fields of Artificial Intelligence and Machine Learning to endow a robot with the ability to not only interact with humans in a social context, but also provide emotional support, in particular for vulnerable populations, such as infants or the elderly. In the past few years, there have been a few research efforts exploring how a robot can socially engage a child from the perspectives of natural language processing and socially assistive robotics. However, none of the proposed methodologies has made use of emotion theory as a core component of the action-planning system. Doing so could enable the robot’s action-planning module to choose those actions that are the most likely to change the affective state of a child in order to improve his/her mood. This work challenges the Interactive Reinforcement Learning paradigm by implementing an interactive decision-planning module developed for RoBoHoN, a social robotic platform that runs on Android OS, with the goal of exploring the feasibility of using a robot to engage with children. Facial features of the patient are captured and processed, determining the emotional reaction of the child to a behavior performed by the robot. Then, these emotions are classified as affective states in a multi-dimensional model. Leveraging the expertise of a child life specialist trainer, the action-planning module interactively learns those actions that are the most appropriate to perform when the child subject is in a specific affective state. In order to validate the usefulness of the proposed methodology, as a first step, we have conducted a pilot study on healthy elementary school aged children. We evaluated the impact of the robot on the participant’s mood through structured questionnaires based on applied pediatric psychology. Our findings show not only that the proposed framework enables a believable and fluid social interaction between child and robot but also that, with our methodology, the robot: • has the ability to change the affective state of children, improving their mood. • is able to establish and maintain social engagement with the child. • can build rapport with the child, as reported by both the child participants and their parents. Based on these results, we believe that our platform has the potential of being implemented using the developed Wizard-of-Oz interface in a real environment (e.g. classroom or hospital), with the potential of releasing a standalone version after the robot has learned the optimal way to act in order to improve the mood of pediatric patients who visit the hospital.