語意一致地自動生成中文新聞的標題

Abstract

在過去幾年裡，很多類神經網路模型，像是各種sequence-to-sequence的變形在標題生成這方面有了很大的進展。Pointer-Generator就是其中一個具有代表性的模型，在機器翻譯、概要、標題生成等等自然語言處理的任務都有不錯的表現。然而，這些模型常常有一個問題，就是生成標題的語意和文章重點的語意不太一致，尤其在中文的任務更是明顯。在這篇論文裡，為了解決上述的問題，我們提出兩個方法加在Pointer-Generator模型上。第一，我們藉由告知模型上一個時間點的上下文語意，改善了注意力機制，讓模型在掌握當前部分的語意後可以注意到其他部分，避免再繼續抄整個句子，那些句子裡可能含有一些比較不重要的資訊。第二，我們針對編碼器加了一些限制，讓他對文章編碼時，可以保有較多的語意資訊。我們將提出的模型應用在兩個台灣新聞報社的中文資料集，結果顯示我們的模型不管是在ROUGE分數上或是人類評估的結果，表現都比Pointer-Generator好。

In the past years, various neural sequence-to-sequence models for title generation have made considerable progresses. Pointer-Generator is one of the representative models recently on many natural language processing tasks, such as machine translation, summarization, title generation and so on. However, these models usually suffer from semantic inconsistency between a generated title and the article, especially for Chinese title generation. In this paper, we propose two methods which augment Pointer-Generator to tackle the issue. First, we improve the attention mechanism by informing model the previous context semantic. By doing so, our model can attend to other parts when it has already got the meanings of the current part. It can prevent model from continuous coping the whole sentence, which probably contains less important information. Second, we add a constraint to the encoder so that it can encode the article with more important semantic information. We apply our model to the two Chinese news datasets collected from two newspaper offices in Taiwan, outperforming Pointer-Generator on both ROUGE scores and the human evaluation.