回憶個人照片之互動式問題提出系統

Abstract

回憶是我們一生當中都一直在從事的活動，我們時常回想過去曾經做過的事情。記憶往往可以作為人們聊天的話題來源，回顧過去的同時也能幫助人們建立自尊心、感受快樂與幸福。在本論文中，我們的目標是開發一個能夠幫助人們從照片中回想起過去回憶的陪伴機器人。我們專注於機器人如何聯想與相片內容相關的概念，並透過詢問相關且吸引人的問題來喚起人們的記憶。為了瞭解照片中的內容，我們應用深度學習的技術來辨識影像中的活動、物體及場景，而後在包含活動常識的馬可夫隨機場中，考慮來自照片及使用者話語的觀察結果，使用循環置信傳播演算法來推斷可能聯想到的概念與話題。之後，機器人根據選擇的話題來提出適當的問題，並在互動中引導使用者回想回憶。最後，我們通過精心設計的實驗評估我們的系統，實驗結果顯示，我們提出的系統能夠提出適當且相關的問題，並且有潛力能夠幫助使用者以有組織性的方式回憶過去。

Reminiscence is a lifelong activity that happens throughout our lifespan. While memories can serve as the topics in people's chit-chat, recalling the past can also help people to build self-esteem and increase the level of happiness. In this thesis, we aim to develop a companion robot that helps people to recollect the memories from the personal photos. We focus on how a robot could associate concepts relevant to the content in the photos and evoke people's memory by asking related and engaging questions. To understand the content in the photo, we apply deep learning techniques to recognize events, objects, and scenes in the image. Then, the observations from the photo and the user utterance are considered in the Markov random field that contains commonsense knowledge of events, whereby the loopy belief propagation is used to infer possible associated concepts and topics. Afterwards, appropriate questions about the selected topics are posed to guide the user to remember the memories in the interaction. Finally, we evaluate our system through well designed experiments. The results show that the proposed system can pose proper and related questions to interact with the user, and has a potential to help and guide the user to recall the past in an organized way.