具備多語言對話、運動輔助、情緒辨識與人類感知導航系統的社交智慧移動機器人之開發

Abstract

隨著科技不斷進步，機器人已從工業應用工具逐漸普及到大眾生活，提供各種日常生活服務。例如，商場中的機器人可提供室內導覽服務，而在戶外則能提供自動駕駛服務，甚至為高齡長輩提供照顧服務。然而，這些服務都仰賴機器人身上的感測器，包括二維或三維光學雷達、 RGB-D相機、麥克風和喇叭等。透過這些感測器，機器人能感知周圍環境的變化，偵測靜態與動態物體的位置，提供安全的導航，引導人們順利到達目的地，同時還能感知人們的情緒和姿態動作，並透過完善的對話系統提供良好的互動體驗。 本論文提出兩個核心系統，以達成以下目標：在人機互動任務中，藉由整合本實驗室先前開發的情緒辨識和運 動輔助系統，結合人臉辨識系統和本論文提出的多語言對話系統及流行的 ChatGPT，完成任務導向和非任務導向的對話任務；而在導航任務中，我們提出了一個基於語意分割和路徑規劃的移動機器人導航系統，以應對動態環境的挑戰。首先，利用本論文所開發的多語言對話系統獲取使用者的目標點，透過 YOLOv8對周圍環境進行物體辨識和語意分割，將辨識出的物體進行動態和靜態分類，結合深度相機或三維光學雷達獲取物體的深度值，構建點雲並進行分群，以獲得物體的三維邊界框，並追蹤這些動態和靜態點雲，使機器人能避免與周圍環境中的移動物體碰撞。接 下來，我們利用本論文提出的 DJPS路徑規劃算法，能快速且高效地獲取全局最佳路徑。最後，結合 改良過的 時間彈性帶 (ModifiedTimed Elastic Band, Modified TEB) 局部軌跡修正，使輪型移動機器人能順利完成局部軌跡修正，使輪型移動機器人能順利完成具有社會認知的導航任務。

With the continuous advancement of technology, robots have gradually transitioned from industrial tools to ubiquitous entities in people's daily lives, providing various services for everyday activities. For instance, robots in shopping malls can offer indoor navigation services, while outdoors they can provide autonomous driving services, and even offer assistance and care for the elderly. However, these services heavily rely on the sensors equipped on the robots, such as 2D or 3D LiDAR, RGB-D cameras, microphones, and speakers. Through these sensors, robots can perceive changes in their surrounding environment, detect the positions of static and dynamic objects, provide safe navigation, guide individuals to their destinations, and also sense people's emotions and gestures, facilitating enhanced interactive experiences through sophisticated dialogue systems. This thesis proposes two core systems to achieve the following objectives: in the human-robot interaction task, a combination of the laboratory's previously developed emotion recognition and exercise assistance systems, together with the facial recognition system and the multilingual dialogue system proposed in this thesis, enables the completion of task-oriented and non-task-oriented dialogue tasks. In the navigation task, a mobile robot navigation system is proposed based on semantic segmentation and path planning to address challenges in dynamic environments. Firstly, the multi-language dialogue system developed in this thesis is utilized to obtain the user's desired destination. The YOLOv8 algorithm is employed for object recognition and semantic segmentation of the surrounding environment. The recognized objects are classified into dynamic and static categories. By integrating a depth camera or 3D lidar, depth information of the objects is acquired, generating point clouds. Clustering of these point clouds enables the extraction of three-dimensional bounding boxes for the objects. Subsequently, the dynamic and static point clouds are tracked to avoid collisions with moving objects in the surrounding environment. Furthermore, the proposed DJPS path planning algorithm efficiently obtains the global optimal path. Finally, the combination of Modified Timed Elastic Band (Modified TEB) approach for local trajectory correction ensures the successful completion of socially aware navigation tasks for wheeled mobile robots.