請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/81331完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 張智星(Jyh-Shing Jang) | |
| dc.contributor.author | Yung-Lin Li | en |
| dc.contributor.author | 李永霖 | zh_TW |
| dc.date.accessioned | 2022-11-24T03:43:39Z | - |
| dc.date.available | 2021-08-20 | |
| dc.date.available | 2022-11-24T03:43:39Z | - |
| dc.date.copyright | 2021-08-20 | |
| dc.date.issued | 2021 | |
| dc.date.submitted | 2021-08-14 | |
| dc.identifier.citation | K. Kowsari, K. Jafari Meimandi, M. Heidarysafa, S. Mendu, L. Barnes, and D.Brown, “Text classification algorithms: A survey,” Information, vol. 10, no.4, p.150, 2019. P. Liu, X. Qiu, and X. Huang, “Recurrent neural network for text classification with multitask learning,” arXiv preprint arXiv:1605.05101, 2016. Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, and E. Hovy, “Hierarchical attention networks for document classification,” in Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, 2016, pp. 1480–1489. S. Lai, L. Xu, K. Liu, and J. Zhao, “Recurrent convolutional neural networks for text classification,” in Twentyninth AAAI conference on artificial intelligence, 2015. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems, 2017, pp. 5998–6008. N. Reimers and I. Gurevych, “Sentencebert: Sentence embeddings using siamese bertnetworks,” arXiv preprint arXiv:1908.10084, 2019. F. Sethi, “Faq (frequently asked questions) chatbot for conversation,” Authorea Preprints, 2020. Z. Yu, Z. Xu, A. W. Black, and A. Rudnicky, “Chatbot evaluation and database expansion via crowdsourcing,” in Proceedings of the chatbot workshop of LREC, vol. 63, 2016, p. 102. G. Salton and C. Buckley, “Termweighting approaches in automatic text retrieval,” Information processing management, vol. 24, no. 5, pp. 513–523, 1988. V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,” arXiv preprint arXiv:1910.01108, 2019. S. Sun, Y. Cheng, Z. Gan, and J. Liu, “Patient knowledge distillation for bert model compression,” arXiv preprint arXiv:1908.09355, 2019. I. Turc, M.W. Chang, K. Lee, and K. Toutanova, “Wellread students learn better: On the importance of pretraining compact models,” arXiv preprint arXiv:1908.08962v2, 2019. D. Jurafsky, Speech language processing. Pearson Education India, 2000. J. Li, A. Sun, J. Han, and C. Li, “A survey on deep learning for named entity recognition,” IEEE Transactions on Knowledge and Data Engineering, 2020. A. Kamath and R. Das, “A survey on semantic parsing,” in 1st Conference on Automated Knowledge Base Construction, AKBC 2019, Amherst, MA, USA, May 20-22, 2019, 2019. [Online]. Available: https://doi.org/10.24432/C5WC7D X. Han, T. Gao, Y. Lin, H. Peng, Y. Yang, C. Xiao, Z. Liu, P. Li, M. Sun, and J. Zhou, “More data, more relations, more context and more openness: A review and outlook for relation extraction,” arXiv preprint arXiv:2004.03186, 2020. B. Fu, Y. Qiu, C. Tang, Y. Li, H. Yu, and J. Sun, “A survey on complex question answering over knowledge base: Recent advances and challenges,” CoRR, vol. abs/2007.13069, 2020. [Online]. Available: https://arxiv.org/abs/2007.13069 S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep learning based text classification: A comprehensive review,” arXiv preprint arXiv:2004.03705, 2020. J. Weizenbaum, “Eliza—a computer program for the study of natural language communication between man and machine,” Communications of the ACM, vol. 9, no. 1, pp. 36–45, 1966. A. Ritter, C. Cherry, and B. Dolan, “Unsupervised modeling of twitter conversations,” in Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2010, pp. 172–180. P. Forchini, “Using movie corpora to explore spoken american english,” Variation and Change in Spoken and Written Discourse: Perspectives from corpus linguistics, vol. 21, p. 123, 2013. L. Shang, Z. Lu, and H. Li, “Neural responding machine for shorttext conversation,” arXiv preprint arXiv:1503.02364, 2015. T. Verma, R. Renu, and D. Gaur, “Tokenization and filtering process in rapidminer,” International Journal of Applied Information Systems, vol. 7, no. 2, pp. 16–18, 2014. B. Pahwa, S. Taruna, and N. Kasliwal, “Sentiment analysisstrategy for text preprocessing,” Int. J. Comput. Appl, vol. 180, pp. 15–18, 2018. V. C. Mawardi, N. Susanto, and D. S. Naga, “Spelling correction for text documents in bahasa indonesia using finite state automata and levinshtein distance method,” in MATEC Web of Conferences, vol. 164. EDP Sciences, 2018, p. 01047. T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” arXiv preprint arXiv:1301.3781, 2013. J. Pennington, R. Socher, and C. Manning, “GloVe: Global vectors for word representation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: Association for Computational Linguistics, Oct. 2014, pp. 1532–1543. [Online]. Available: https://www.aclweb.org/anthology/D141162 P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” Transactions of the Association for Computational Linguistics, vol. 5, pp. 135–146, 2017. G. P. Sanjay, V. Nagori, G. Sanjay, and V. Nagori, “Comparing existing methods for predicting the detection of possibilities of blood cancer by analyzing health data,” Int. J. Innov. Res. Sci. Technol, vol. 4, pp. 10–14, 2018. A. Shrikumar, P. Greenside, and A. Kundaje, “Learning important features through propagating activation differences,” arXiv preprint arXiv:1704.02685, 2017. J. Devlin, M.W. Chang, K. Lee, and K. Toutanova, “Bert: Pretraining of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018. J. Bromley, J. W. Bentz, L. Bottou, I. Guyon, Y. LeCun, C. Moore, E. Säckinger, and R. Shah, “Signature verification using a“siamese'time delay neural network,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 7, no. 04, pp. 669–688, 1993. N. Thakur, N. Reimers, J. Daxenberger, and I. Gurevych, “Augmented sbert: Data augmentation method for improving biencoders for pairwise sentence scoring tasks,” arXiv preprint arXiv:2010.08240, 2020. Z. Zhao, W. Zhang, W. Che, Z. Chen, and Y. Zhang, “An evaluation of chinese human-computer dialogue technology,” Data Intelligence, vol. 1, no. 2, pp. 187–200, 2019. X. Zhang, J. Zhao, and Y. LeCun, “Characterlevel convolutional networks for text classification,” arXiv preprint arXiv:1509.01626, 2015. | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/81331 | - |
| dc.description.abstract | "常見問題集(frequently asked questions, FAQ)是在業務場景中客戶最常問的問題集合，本篇論文在建立一有效回答常見問題集的聊天機器人(chatbot)。首先，問題的答案經常會隨著時間而改變，為了語料的穩定性和模型建立的準確性，我們將回答 FAQ 的問題轉變為從候選中檢索出最合適的匹配對象。接著，我們使用 term frequency–inverse document frequency (TFIDF)作為聊天機器人檢索匹配對象的根據，我們發現到 TFIDF 並不能識別客戶對同一個標準問題所產生出的不同測試題(query)。所以我們提出使用 BERT 來提升模型識別問題語義的能力，我們探討了使用不同比對模式來微調 BERT的情況，我們的結果超越了傳統上使用 BERT 對 query 進行文本分類的結果。同時我們比較text classification with BERT、cross-encoder BERT、Siamese BERT，在小資料量資料集例如：公司常見問題集，準確率從text classification with BERT 的74.20%和Siamese BERT的74.50%提升到cross-encoder BERT的81.00%。但是在大資料量資料集例如：Yahoo! Answers，text classification with BERT則有最高的準確率。另外，我們使用了不同的資料擴增方法，reverse pair和繁簡增生在cross-encoder BERT上都能提高準確率。" | zh_TW |
| dc.description.provenance | Made available in DSpace on 2022-11-24T03:43:39Z (GMT). No. of bitstreams: 1 U0001-1907202110333700.pdf: 3717310 bytes, checksum: 607be4c02a9e4db2fabaa2c9e7a7e2ae (MD5) Previous issue date: 2021 | en |
| dc.description.tableofcontents | Acknowledgements iii 摘要 v Abstract vii Contents ix List of Figures xiii List of Tables xv Chapter 1 Introduction 1 1.1 研究目的與動機. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 研究主題. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.3 章節概要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Chapter 2 Related Work 3 2.1 背景介紹. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 聊天機器人. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.3 文本分類. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.4 詞權重. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.5 神經網路. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.6 BERT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Chapter 3 Datasets Methods 17 3.1 資料集介紹. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.1.1 公司常見問題集. . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.1.2 SMPECDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.1.3 Yahoo! Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2 問題定義. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3 模型介紹. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.3.1 TFIDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.3.2 Text classification with BERT . . . . . . . . . . . . . . . . . . . . 21 3.3.3 Siamese BERT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.3.4 Crossencoder BERT . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.3.5 資料擴增方法. . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Chapter 4 Experiments 29 4.1 硬體規格和訓練參數. . . . . . . . . . . . . . . . . . . . . . . . . 29 4.2 實驗設計. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.3 實驗一：TFIDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.4 實驗二：不同的輸入問題對比較. . . . . . . . . . . . . . . . . . 33 4.5 實驗三：資料擴增與負採樣. . . . . . . . . . . . . . . . . . . . . 38 4.6 實驗四：資料數量不足的結果. . . . . . . . . . . . . . . . . . . . 40 4.7 實驗五：不同模型間的正確率比較. . . . . . . . . . . . . . . . . 43 Chapter 5 Conclusions Future work 47 5.1 結論. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.2 未來展望. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 References 49 | |
| dc.language.iso | zh-TW | |
| dc.subject | BERT | zh_TW |
| dc.subject | 常見問題集 | zh_TW |
| dc.subject | 聊天機器人 | zh_TW |
| dc.subject | 問題相似度 | zh_TW |
| dc.subject | 問題回答 | zh_TW |
| dc.subject | Question Answering | en |
| dc.subject | BERT | en |
| dc.subject | FAQ | en |
| dc.subject | Chatbot | en |
| dc.subject | Question Similarity | en |
| dc.title | 使用深度學習的FAQ聊天機器人：實作與比較 | zh_TW |
| dc.title | Construction of Frequently Asked Questions Chatbot with Deep Learning : Implementation and Comparison | en |
| dc.date.schoolyear | 109-2 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 陳永耀(Hsin-Tsai Liu),陳縕儂(Chih-Yang Tseng) | |
| dc.subject.keyword | 常見問題集,聊天機器人,問題相似度,問題回答,BERT, | zh_TW |
| dc.subject.keyword | FAQ,Chatbot,Question Similarity,Question Answering,BERT, | en |
| dc.relation.page | 53 | |
| dc.identifier.doi | 10.6342/NTU202101558 | |
| dc.rights.note | 同意授權(限校園內公開) | |
| dc.date.accepted | 2021-08-16 | |
| dc.contributor.author-college | 電機資訊學院 | zh_TW |
| dc.contributor.author-dept | 資訊工程學研究所 | zh_TW |
| 顯示於系所單位： | 資訊工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| U0001-1907202110333700.pdf 授權僅限NTU校內IP使用（校園外請利用VPN校外連線服務） | 3.63 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。