請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/79985完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 許永真(Jung-Jen Hsu) | |
| dc.contributor.author | Wei-Fan Chang | en |
| dc.contributor.author | 張葳凡 | zh_TW |
| dc.date.accessioned | 2022-11-23T09:19:44Z | - |
| dc.date.available | 2021-08-10 | |
| dc.date.available | 2022-11-23T09:19:44Z | - |
| dc.date.copyright | 2021-08-10 | |
| dc.date.issued | 2021 | |
| dc.date.submitted | 2021-07-23 | |
| dc.identifier.citation | [1] R. Bhagat and E. Hovy. Squibs: What is a paraphrase? Computational Linguistics, 2013. [2] I. Bolshakov and A. Gelbukh. Synonymous paraphrasing using wordnet and internet. In Natural Language Processing and Information Systems. Springer Berlin Heidelberg, 2004. [3] Z. Cao, C. Luo, W. Li, and S. Li. Joint copying and restricted generation for paraphrase. In Proceedings of the ThirtyFirst AAAI Conference on Artificial Intelligence. AAAI Press, 2017. [4] M. Chen, Q. Tang, S. Wiseman, and K. Gimpel. Controllable paraphrase generation with a syntactic exemplar. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2019. [5] W. Chen, J. Tian, L. Xiao, H. He, and Y. Jin. A semantically consistent and syntactically variational encoder-decoder framework for paraphrase generation. In Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 2020. [6] Y.C. Chen, Z. Gan, Y. Cheng, J. Liu, and J. Liu. Distilling knowledge learned in BERT for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020. [7] A. CONNEAU and G. Lample. Crosslingual language model pretraining. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2019. [8] J. Devlin, M.W. Chang, K. Lee, and K. Toutanova. BERT: Pretraining of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 2019. [9] L. Dong, J. Mallinson, S. Reddy, and M. Lapata. Learning to paraphrase for question answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2017. [10] Y. Fu, Y. Feng, and J. P. Cunningham. Paraphrase generation with latent bag of words. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2019. [11] A. Gupta, A. Agarwal, P. Singh, and P. Rai. A deep generative framework for paraphrase generation. In Proceedings of the ThirtySecond AAAI Conference on Artificial Intelligence. AAAI Press, 2018. [12] S. Hochreiter and J. Schmidhuber. Long short term memory. Neural computation, 1997. [13] E. Kacupaj, B. Banerjee, K. Singh, and J. Lehmann. Paraqa: A question answering dataset with paraphrase responses for singleturn conversation. In R. Verborgh, K. Hose, H. Paulheim, P.A. Champin, M. Maleshkova, O. Corcho, P. Ristoski, and M. Alam, editors, The Semantic Web. Springer International Publishing, 2021. [14] D. Kauchak and R. Barzilay. Paraphrasing for automatic evaluation. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference. Association for Computational Linguistics, 2006. [15] D. P. Kingma and M. Welling. Autoencoding variational bayes, 2014. [16] A. Kumar, S. Bhattamishra, M. Bhandari, and P. Talukdar. Submodular optimizationbased diverse paraphrasing and its effectiveness in data augmentation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 2019. [17] Z. Li, X. Jiang, L. Shang, and H. Li. Paraphrase generation with deep reinforcement learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2018. [18] Z. Li, X. Jiang, L. Shang, and Q. Liu. Decomposable neural paraphrase generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2019. [19] C.Y. Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out. Association for Computational Linguistics, 2004. [20] M. Liu, E. Yang, D. Xiong, Y. Zhang, Y. Meng, C. Hu, J. Xu, and Y. Chen. A learning exploring method to generate diverse paraphrases with multiobjective deep reinforcement learning. In Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 2020. [21] G. A. Miller. Wordnet: A lexical database for english. Commun. ACM, 1995. [22] K. Papineni, S. Roukos, T. Ward, and W.J. Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2002. [23] A. Prakash, S. A. Hasan, K. Lee, V. Datla, A. Qadir, J. Liu, and O. Farri. Neural paraphrase generation with stacked residual LSTM networks. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. The COLING 2016 Organizing Committee, 2016. [24] N. Reimers and I. Gurevych. SentenceBERT: Sentence embeddings using Siamese BERT networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP). Association for Computational Linguistics, 2019. [25] K. Song, X. Tan, T. Qin, J. Lu, and T.Y. Liu. Mass: Masked sequence to sequence pretraining for language generation. In International Conference on Machine Learning, 2019. [26] I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2014. [27] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2017. [28] X. Wu, S. Lv, L. Zang, J. Han, and S. Hu. Conditional bert contextual augmentation. In Computational Science – ICCS 2019. Springer International Publishing, 2019. [29] J. Yang, M. Wang, H. Zhou, C. Zhao, W. Zhang, Y. Yu, and L. Li. Towards making the most of bert in neural machine translation. Proceedings of the AAAI Conference on Artificial Intelligence, 2020. [30] H. Zhang, J. Cai, J. Xu, and J. Wang. Pretraining based natural language generation for text summarization. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). Association for Computational Linguistics, 2019. [31] T. Zhang*, V. Kishore*, F. Wu*, K. Q. Weinberger, and Y. Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, 2020. [32] W. Zhou, T. Ge, K. Xu, F. Wei, and M. Zhou. BERT based lexical substitution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2019. [33] Z. Zhou, M. Sperber, and A. Waibel. Paraphrases as foreign languages in multilingual neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. Association for Computational Linguistics, 2019. | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/79985 | - |
| dc.description.abstract | 產生換句話說的句子是重要的自然語言處理任務，生成的句子必須傳達相同的意思，但是使用不同用詞。由於近期預訓練語言模型有強大的自然語言理解能力，這篇論文中我們提出有效利用BERT 模型的方法來探索語意相近的用詞，使用在換句話說句子的生成中。雖然利用BERT 做文字代換是常見的資料增強方法，但在換句話說的任務上帶來的進步幅度是有限的，因此我們不直接從分佈中隨機抽樣詞來做代換。由於一些相關研究的啟發，我們提出的方法將BERT 生成的分佈作為潛在的向量，整合在在Transformer Decoder 的注意力機制中。實驗的結果顯示我們提出的方法的BLEU 分數比現階段表現最好的模型更高，比起基準模型也得到更好的BLEU 與Rouge 分數，透過分析我們更觀察到提出的模型使用了訓練資料中沒有的詞彙代換來生成換句話說的句子。因此透過利用BERT 所學到的知識，我們能生成以量化的指標衡量下，表現更好的換句話說。 | zh_TW |
| dc.description.provenance | Made available in DSpace on 2022-11-23T09:19:44Z (GMT). No. of bitstreams: 1 U0001-2107202117333400.pdf: 1640155 bytes, checksum: 6935b5861f54b67c6c1a8c353f2fc729 (MD5) Previous issue date: 2021 | en |
| dc.description.tableofcontents | 口試委員審定書i 致謝iii 摘要v Abstract vii Contents ix List of Figures xi List of Tables xiii Chapter 1 Introduction 1 1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Research Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Chapter 2 Related Work 5 2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1.1 Different Levels of Granularity . . . . . . . . . . . . . . . . . . . . 6 2.2 Utilize Pre-Trained Language Model for Text Generation . . . . . . . 7 Chapter 3 Problem Definition 9 3.1 Paraphrase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Chapter 4 Methodology 11 4.1 Seq2seq Generation . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.3 The Latent Vector from BERT . . . . . . . . . . . . . . . . . . . . . 13 4.4 The Latent BERT Vector in the Decoder . . . . . . . . . . . . . . . . 14 Chapter 5 Experiments 17 5.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.2 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.3 Experiment Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.4 Compared Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 5.4.1 Compared with Lexical-Substitution Based Methods . . . . . . . . 19 5.4.2 Compared with Related Work and State-of-the-Arts . . . . . . . . . 20 Chapter 6 Results and Analysis 23 6.1 Evaluation Metrics Results . . . . . . . . . . . . . . . . . . . . . . . 23 6.1.1 Compared with Lexical-Substitution Based Methods . . . . . . . . 23 6.1.2 Compared with Related Work and State-of-the-Arts . . . . . . . . . 24 6.2 Training Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 6.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 6.3.1 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 6.3.2 The Latent BERT Vector . . . . . . . . . . . . . . . . . . . . . . . 29 6.3.3 The Dimension Reduction Linear Layer . . . . . . . . . . . . . . . 30 Chapter 7 Conclusion 33 7.1 Summary and Contributions . . . . . . . . . . . . . . . . . . . . . . 33 7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Bibliography 35 | |
| dc.language.iso | en | |
| dc.subject | 類神經網路 | zh_TW |
| dc.subject | 換句話說生成 | zh_TW |
| dc.subject | 自然語言處理 | zh_TW |
| dc.subject | 預訓練語言模型 | zh_TW |
| dc.subject | Paraphrase Generation | en |
| dc.subject | BERT | en |
| dc.subject | Natural Language Processing | en |
| dc.subject | Neural Network | en |
| dc.title | 利用預訓練語言遮罩模型探索語意相近用詞於換句話說生成之研究 | zh_TW |
| dc.title | Utilizing BERT to Explore Semantically Similar Words for Paraphrase Generation | en |
| dc.date.schoolyear | 109-2 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 李宏毅(Hsin-Tsai Liu),陳縕儂(Chih-Yang Tseng),陳信希,蔡宗翰 | |
| dc.subject.keyword | 換句話說生成,預訓練語言模型,自然語言處理,類神經網路, | zh_TW |
| dc.subject.keyword | Paraphrase Generation,BERT,Natural Language Processing,Neural Network, | en |
| dc.relation.page | 39 | |
| dc.identifier.doi | 10.6342/NTU202101638 | |
| dc.rights.note | 同意授權(全球公開) | |
| dc.date.accepted | 2021-07-23 | |
| dc.contributor.author-college | 電機資訊學院 | zh_TW |
| dc.contributor.author-dept | 資訊工程學研究所 | zh_TW |
| 顯示於系所單位： | 資訊工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| U0001-2107202117333400.pdf | 1.6 MB | Adobe PDF | 檢視/開啟 |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。