雙層語意控制之自然語言理解與自動對話摘要

Abstract

本篇論文旨在探討自然語言處理領域中的兩個問題:自然語言理解及自動對話摘要。現有的意圖識別(intent prediction)及槽填充(slot filling)模型共用帶有注意力(attention)機制的遞歸神經網路(recurrent neural network)，然而其注意力權重的部分依然分成意圖及槽兩個部分。考量到意圖及槽具有高度相關性，本篇論文提出一個語義控制閘(slot gate)學習意圖注意力向量及槽注意力向量的關係，以提升模型的準確度。實驗結果證明，我們提出的模型可以在三個資料集(ATIS, Snips and AMI)上，比現有的模型顯著地進步。 此外，語意控制的概念是否也可以進一步延伸至多句的情境？本篇論文選擇討論文章摘要的問題。過去討論抽象式摘要(abstractive summarization)問題時，主要使用單一敘述者的資料（新聞、學術刊物等）。在一個對話中，存在複數參與者彼此之間的互動，這被稱爲對話行爲(dialogue act)，而這些互動的資訊可能可以幫助產生品質更好的自動對話摘要。本篇論文透過加入一個句子控制閘(sentence gate)來學習對話行爲及對話摘要的關係，使類神經對話摘要模型獲取對話行爲的資訊。實驗結果證明，我們提出的模型可以在AMI資料集上，比現有的模型顯著地進步。

Attention-based recurrent neural network models for joint intent detection and slot filling have achieved the state-of-the-art performance, while they have independent attention weights. Considering that slot and intent have the strong relationship, this thesis proposes a slot gate that focuses on learning the relationship between intent and slot attention vectors in order to obtain better semantic frame results by the global optimization. The experiments show that proposed model significantly improves the performance compared to the state-of-the-art baselines on benchmark ATIS, Snips and AMI datasets. Furthermore, can we extend the gating mechanism to multi-sentences? In this thesis, we choose the summarization task. Abstractive summarization has been widely studied, while the prior work mainly focused on summarizing single-speaker documents (news, scientific publications etc). In dialogues, there are different interactions between speakers, which is usually defined as dialogue acts. These interactive signals may provide informative cues for better summarizing dialogues. This thesis aims to leverage dialogue acts in a neural summarization model, where a sentence gate is designed to model the relationship between dialogue acts and summaries. The experiments show that proposed model significantly improves the abstractive summarization performance compared to the state-of-the-art baselines on AMI meeting corpus, demonstrating the usefulness of interactive signals.