使用深層強化學習之互動式語音數位內容檢索

Abstract

本論文之主軸在探討語音數位內容之互動式檢索(Interactive Retrieval of Spoken Content)。近年來多媒體數位內容(Multimedia Content) 如線上課程、影音節目、會議錄音等大幅增加，語音數位內容( Spoken Content ) 之檢索也因而大受重視。本論文之目標放在互動式檢索； 由於語音或多媒體文件很難呈現在螢光幕上故瀏覽耗時，而過差的語音辨識率更可能使檢索結果不如人意，因此藉由系統與使用者互動使系統對使用者想找的資訊有更多瞭解，是一個有效改善此一問題的方法。 在本論文中，我們不但模仿前人用馬可夫決策模型(Markov Decision Process，MDP)來建立互動式檢索的模型，採用強化學習(Reinforcement Learning)演算法來學習出最佳系統決策， 更使用深層強化學習(Deep Reinforcement Learning)解決問題讓整個技術向前邁進一大步。實驗顯示，我們提出的方法確實能夠大幅改善檢索程序，幫助使用者更有效的找到所要找的資訊。

Interactive retrieval is important for spoken content. The reason is because when looking for text documents, one can easily scan through and select on a search engine result page, whereas similar privileges don not exist when searching for spoken content. Besides, it is hard for the users to find the desired spoken content when the search results are noisy, which usually happens due to the imperfect speech recognition components in spoken content retrieval. A way to counter the difficulties of spoken content retrieval is human-machine interaction that machine takes different actions to request additional information from the user to obtain better retrieval results. The most suitable actions depend on the situations, so in previous works, some hand-crafted states estimated from the current search results are used to determine the actions, but the hand-crafted states are not necessary the best indicator for choosing actions. In this paper, we applied the Deep-Q- Learning method in interactive retrieval of spoken content. Deep-Q- Learning sidesteps the estimation of the hand-crafted states and can directly determine the action based on retrieval results without any human knowledge. It reached discernible improvements compared with the hand-crafted states.