請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/16055
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 顏嗣鈞(Hsu-Chun Yen) | |
dc.contributor.author | Daniel Li | en |
dc.contributor.author | 李德堯 | zh_TW |
dc.date.accessioned | 2021-06-07T17:59:28Z | - |
dc.date.copyright | 2020-08-21 | |
dc.date.issued | 2020 | |
dc.date.submitted | 2020-08-04 | |
dc.identifier.citation | [1] I. G. Israel, “Information explosion and university libraries : Current trends and strategies for intervention,” 2010. [2] K. Knight, “Automatic language translation generation help needs badly,” In MT Summit XI Work- shop on Using Corpora for NLG: Keynote Address, pp. 5–8, 2008. [3] A. Ramanathan, H. Choudhary, A. Ghosh, and P. Bhattacharyya, “Case markers and morphology: addressing the crux of the fluency problem in English-Hindi smt,” In Proceedings of ACL-IJCNLP, 2009. [4] Y. Zhang and S. Clark, “SyntaxBased Grammaticality Improvement using CCG and Guided Search,” In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1147–1157, 2011. [5] T.R.Gruber, “A translation approach to portable ontology specifications,” Knowledge acquisition, vol. 5, no. 2, pp. 199–220, 1993. [6] M.-C.Chung,“A study and construction on ontology-based document classification web service,” Master Thesis, Department of Computer Science and Information Engineering (CSIE), National Cheng Kung University, Tainan City, 2004. [7] C.-S. Lee, C.-P. Chen, H.-J. Chen, and Y.-H. Kuo, “A fuzzy classification agent for personal e-news service,” International Journal of Fuzzy Systems, vol. 4, no. 4, 2002. [8] C.-S. Sung, “詞網同義詞集的中文語意表達之研究,” Master Thesis, Department of Computer Science, Soochow University, Taipei City, 2003. [9] Y. Liu, Y. Zhang, W. Che, and B. Qin, “Transition-based syntactic linearization,” In NAACL, 2015 [10] M. Horvat and W. Byrne, “A graph-based approach to string regeneration,” . In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 85–95, Gothenburg, Sweden, April. Association for Computational Linguistics, 2014. [11] D. L. Applegate, R. E. Bixby, V. C. atal, and W. J. Cook, “ Concorde TSP Solver,” 2006 [12] A. M. R. Allen Schmaltz and S. M. Shieber, “Word ordering without syntax,” In EMNLP, 2016. [13] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” In Proceedings of NIPS, 2014. [14] E. Hasler, F. Stahlberg, M. Tomalin, A. de Gispert, and B. Byrne, “ A comparison of neural models for word ordering,” In Proceedings of the International Natural Language Generation Conference, Santiago de Compostela, Spain, 2017. [15] J. Elman, “Finding structure in time,” Cognitive Science, vol. 14, no. 2, pp. 179–211, 1990. [16] S. Hochreiter and J. Schmidhuber, “Long Short Term Memory. Neural Computation,” Neural computation,9(8):1735–1780, 1997. [17] H. Sak, A. Senior, and F. Beaufays., “Long short-term memory recurrent neural network architectures for large scale acoustic modeling,” Proceedings of the Annual Conference of International Speech Communication Association (INTERSPEECH), 2014. [18] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. G. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems, pp. 5998–6008, 2017. [19] J. P. Turian, L. Ratinov, and Y. Bengio, “Word representations: A simple and general method for semi-supervised learning,” ACL 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics,July11-16,2010,Uppsala, Sweden, pp. 384–394, 2010. [20] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” CCoRR, vol. abs/1301.3781, 2013. [21] M.Peters, M.Neumann, M.Iyyer, M.Gardner, C.Clark, K.Lee, and L.Zettlemoyer, “Deep contextualized word representations,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2227–2237, 2018. [22] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019. [23] S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning, “A large annotated corpus for learning natural language inference,” Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, (Lisbon, Portugal), pp. 632– 642, Association for Computational Linguistics, Sept. 2015. [24] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ questions for machine comprehension of text,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, (Austin, Texas), pp. 2383–2392, Association for Computational Linguistics, Nov. 2016. [25] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank,” Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, (Seattle, Washington, USA), pp. 1631–1642, Association for Computational Linguistics, Oct. 2013. [26] E. F. Tjong Kim Sang and F. D. Meulder, “Introduction to the conll-2003 shared task: Language-independent named entity recognition,” Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, CONLL ’03,(Stroudsburg, PA, USA), pp. 142–147, Association for Computational Linguistics, 2003. [27] Y. Hsieh, Y. Chang, Y. Huang, C. C. S. Yeh, and W. Hsu, “MONPA: multi- objective named-entity and part-of-speech annotator for chinese using recurrent neural network,” in Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, vol. 2, pp. 80–85, 2017. [28] Y. Meng, X. Li, X. Sun, Q. Han, A. Yuan, and J. Li, “Is word segmentation necessary for deep learning of chinese representations? ,” arXiv preprint arXiv:1905.05526, 2019. [29] M. Ballesteros, C. Dyer, and N. A. Smith, “ Improved transition-based dependency parsing by modeling characters instead of words with LSTMs ,” In Proceedings of EMNLP, 2015. [30] W. Ling, C. Dyer, A. W. Black, I. Trancoso, R. Fermandez, S. Amir, L. Marujo, and T. Lu´ıs, “ Finding function in form: Compositional character models for open vocabulary word representation ,” In EMNLP, 2015. [31] Y. Kim, Y. Jernite, D. Sontag, and A. M. Rush, “ Character-aware neural language models ,” In AAAI, 2016. [32] Hinton, G. E, Srivastava, K. Nitish, Alex, Sutskever, Ilya, Salakhutdinov, and R. R, “Improving neural networks by preventing co-adaptation of feature detectors,” arXiv preprint arXiv:1207.0580, 2012b. [33] Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting.” . J. Mach. Learn. Res., 15(1):1929–1958, 2014. [34] Y. Shen, Z. Lin, C.-W. Huang, and A. Courville, “ Neural language modeling by jointly learning syntax and lexicon,” arXiv preprint arXiv:1711.02013,, 2017. [35] 謝佑明, 楊敦淇, and 陳克健, “語法規律的抽取及普遍化與精確化的研究,” Proceedings of ROCLING XVI, pp.141-150, 2004. [36] Y. Cui, W. Che, T. Liu, B. Qin, Z. Yang, S. Wang, and G. Hu, “Pre-training with whole word masking for chinese bert,” arXiv preprint arXiv:1906.08101, 2019. [37] K. Guu, T. B. Hashimoto, Y. Oren, and P. Liang., “Generating sentences by editing prototypes,” Transactions of the Association for Computational Linguistics (TACL) 0, 2018. [38] K. Papineni, S. Roukos, T. Ward, and W. Zhu, “Bleu: a method for automatic evaluation of machine translation,” In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA, July. Association for Computational Linguistics., 2002. | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/16055 | - |
dc.description.abstract | 句子重組問題是指將通順的句子隨機排序打亂後,轉成詞袋模型,並將其中的詞彙重組,還原回一個通順的句子。此目的是提高機器生成的文本的語法性和流暢性,儘管句子重組問題在英文自然語言處理上已有多篇相關研究,但在中文自然語言處理中尚未有相關研究。 本論文基於英文上的相關研究,進行語言模型的更改,以及有別於以往集束搜索的方式,透過從訓練語料中建立詞類模板有效降低搜尋成本,使其和我們使用的語言模型BERT更為融合,在中文的樹庫資料集中拿到了0.82的BLEU分數。並在研究過程中訓練出一個EHowNet分類器,透過BERT詞向量的群聚,將其投影到正確的類別,可以有效解決資料庫中out-of-vocabulary的問題,對基於知識的自然語言處理有很大的幫助。 | zh_TW |
dc.description.abstract | The problem of so-called sentence reordering means to randomly scramble the sequence of an orderly sentence and transforms it into a bag of words model among which it ends up with the tokens permutated and restored to an orderly way. The purpose is to increase the fluency and grammatical structure of the sentence generated by machines. In fact, there have been plenty of researches in English in terms of sentence reordering and natural language processing (NLP) but not in the scope of Chinese language. This paper hinges on the study of related works in English and is aimed at modification of language model to differentiate the existing method of beam search. Through the process of training corpus to establish POS patterns can effectively reduce the search efforts. By seamlessly combining it with BERT language model, it got a high mark of 0.82 BLEU points on Chinese Treebank. By virtue of creating the EHowNet classifier incorporating with the cluster of BERT word vectors, the system can project the accurate category and successfully address the issue of out-of-vocabulary in the database and did a great help in the knowledge-based NLP. | en |
dc.description.provenance | Made available in DSpace on 2021-06-07T17:59:28Z (GMT). No. of bitstreams: 1 U0001-3107202002091400.pdf: 6948036 bytes, checksum: 982892d5490b6e5565c7fdb87a3c8776 (MD5) Previous issue date: 2020 | en |
dc.description.tableofcontents | 誌謝 ii 摘要 iv Abstract vi List of Figures xi List of Tables xiii Chapter 1 緒論 1 1.1 研究目的與動機 1 1.2 章節概要 3 Chapter 2 問題描述 4 2.1 詞袋模型(Bag-of-Words model) 4 2.2 名詞片語(Noun Phrase) 4 2.3 句子重新排序 5 Chapter 3 相關研究 6 3.1 本體論 6 3.2 廣義知網 7 3.3 句子重組 7 3.3.1 基於語法結構的句子重組 8 3.3.2 基於圖學和語言模型的句子重組 9 3.3.3 基於神經網路的句子重組 11 Chapter 4 語言模型與詞向量 13 4.1 語言模型(Language Model)與網路訓練架構 13 4.1.1 n元語法(n-gram) 13 4.1.2 遞歸神經網路(RNN) 14 4.1.3 長短期記憶神經網路(LSTM) 14 4.1.4 Transformer 15 4.2 詞向量(Word Representation) 16 4.2.1 One-Hot representation 17 4.2.2 Word2Vec 17 4.2.3 Deep contextualized word representations(ELMo) 18 4.2.4 BERT 19 Chapter 5 句子重組模型 21 5.1 中文語料處理 21 5.2 句子概念模板 21 5.2.1 EHowNet未知詞處理以及詞義消歧 22 5.2.2 EHowNet概念分類器 23 5.2.3 基於概念的n元語法模型 25 5.2.4 基於BERT的句子重組模型 27 5.2.5 以詞類模板降低搜尋成本 29 Chapter 6 實驗結果與討論 32 6.1 EHowNet分類器 32 6.1.1 資料集和評估方式 32 6.1.2 準確度 34 6.1.3 錯誤分析與討論 36 6.2 句子重組 37 6.2.1 資料集 37 6.2.2 評估方式 38 6.2.3 準確度 39 6.2.4 錯誤分析與討論 42 Chapter 7 結論與未來工作 45 7.1 結論 45 7.2 未來展望 45 Bibliography 47 Appendix A 中研院平衡語料庫詞類標記 52 | |
dc.language.iso | zh-TW | |
dc.title | 準則式中文句子重組還原 | zh_TW |
dc.title | A Principle Based Approach for Chinese Word Reordering | en |
dc.type | Thesis | |
dc.date.schoolyear | 108-2 | |
dc.description.degree | 碩士 | |
dc.contributor.coadvisor | 許聞廉(Wen-Lian Hsu) | |
dc.contributor.oralexamcommittee | 張詠淳(Yung-Chun Chang) | |
dc.subject.keyword | 語意理解,自然語言處理,句子重組,語言模型,廣義知識本體, | zh_TW |
dc.subject.keyword | Semantic Understanding,NLP,Word Reordering,Language Model,EHowNet, | en |
dc.relation.page | 54 | |
dc.identifier.doi | 10.6342/NTU202002141 | |
dc.rights.note | 未授權 | |
dc.date.accepted | 2020-08-05 | |
dc.contributor.author-college | 電機資訊學院 | zh_TW |
dc.contributor.author-dept | 電機工程學研究所 | zh_TW |
顯示於系所單位: | 電機工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
U0001-3107202002091400.pdf 目前未授權公開取用 | 6.79 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。