利用多查詢記憶網路學習詞袋文件表達法

Abstract

文本向量能提供有效且基於統計的資訊，並廣泛地被應用在文字領域，例如：網頁搜尋、問答系統、文本相似度上。大部份既有的方法以文本詞頻作為特徵，並依靠詞向量衡量全域性的重要性。然而，文章中詞的重要性不只需要考慮詞頻以及全域性的重要性，也必須考慮該文章本身的含義。在這篇論文中，我們提出一個基於注意力機制的非監督式預測模型來衡量每個字在文本中的重要性。並且，考慮一個文本能有多種解釋方式，我們使用多查詢記憶網路來抽取各種面向的文本含義，並使用循環式及門來匯集各種含義。最後，我們使用了兩個公開資料集去驗證我們的模型，實驗結果顯示我們的方法能夠顯著的超越相關方法中的頂尖技術。

Document representation provide essential statistical information compressed features for many tasks in the text domain, e.g., web search, question answering, document similarity and relevance judgement. Current methods use term frequencies as local features and rely on word embeddings to measure the global importance. However, the importance of words in a document might depend on the meaning of the document and can not globally measured. In this work, we propose an attention-based unsupervised predictive model to reweight the importance of words in a document. Also, considering the multiple interpretations of a single document, we multi-queries memory networks to extract the semantics in different views. And we use recurrent and gating method to combine the semantics. The experimental results show our proposed model outperforms the state-of-the-art works on two benchmark datasets.