Please use this identifier to cite or link to this item:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/87268Full metadata record
| ???org.dspace.app.webui.jsptag.ItemTag.dcfield??? | Value | Language |
|---|---|---|
| dc.contributor.advisor | 陳縕儂 | zh_TW |
| dc.contributor.advisor | Yun-Nung Chen | en |
| dc.contributor.author | 黃千芝 | zh_TW |
| dc.contributor.author | Chien-Chi Huang | en |
| dc.date.accessioned | 2023-05-18T16:44:23Z | - |
| dc.date.available | 2023-11-09 | - |
| dc.date.copyright | 2023-05-11 | - |
| dc.date.issued | 2023 | - |
| dc.date.submitted | 2023-02-18 | - |
| dc.identifier.citation | [1] E. Choi, H. He, M. Iyyer, M. Yatskar, W.-t. Yih, Y. Choi, P. Liang, and L. Zettle- moyer. Quac: Question answering in context. arXiv preprint arXiv:1808.07036, 2018.
[2] M. Del Tredici, G. Barlacchi, X. Shen, W. Cheng, and A. de Gispert. Ques- tion rewriting for open-domain conversational qa: Best practices and limitations. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 2974–2978, 2021. [3] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. [4] A. Elgohary, D. Peskov, and J. Boyd-Graber. Can you unpack that? learning to rewrite questions-in-context. Can You Unpack That? Learning to Rewrite Questions-in-Context, 2019. [5] M. Grbovic, N. Djuric, V. Radosavljevic, F. Silvestri, and N. Bhamidipati. Context- and content-aware embeddings for query rewriting in sponsored search. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 383–392, 2015. [6] J. Gu, Z. Lu, H. Li, and V. O. Li. Incorporating copying mechanism in sequence-to- sequence learning. arXiv preprint arXiv:1603.06393, 2016. [7] M. Huang, F. Li, W. Zou, and W. Zhang. Sarg: A novel semi autoregressive gen- erator for multi-turn incomplete utterance restoration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13055–13063, 2021. [8] Y. Ju, F. Zhao, S. Chen, B. Zheng, X. Yang, and Y. Liu. Technical report on conver- sational question answering. arXiv preprint arXiv:1909.10772, 2019. [9] Q. Liu, B. Chen, J.-G. Lou, B. Zhou, and D. Zhang. Incomplete utterance rewriting as semantic segmentation. arXiv preprint arXiv:2009.13166, 2020. [10] Y. Ohsugi, I. Saito, K. Nishida, H. Asano, and J. Tomita. A simple but effective method to incorporate multi-turn context with bert for conversational machine com- prehension. arXiv preprint arXiv:1905.12848, 2019. [11] Y.PapakonstantinouandV.Vassalos.Queryrewritingforsemistructureddata.ACM SIGMOD Record, 28(2):455–466, 1999. [12] M.Qiu,X.Huang,C.Chen,F.Ji,C.Qu,W.Wei,J.Huang,andY.Zhang.Reinforced history backtracking for conversational question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13718–13726, 2021. [13] C. Qu, L. Yang, C. Chen, M. Qiu, W. B. Croft, and M. Iyyer. Open-retrieval conver- sational question answering. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 539–548, 2020. [14] C. Qu, L. Yang, M. Qiu, Y. Zhang, C. Chen, W. B. Croft, and M. Iyyer. Attentivehistory selection for conversational question answering. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1391–1400, 2019. [15] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. [16] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1–67, 2020. [17] S. Rizvi, A. Mendelzon, S. Sudarshan, and P. Roy. Extending query rewriting tech- niques for fine-grained access control. In Proceedings of the 2004 ACM SIGMOD international conference on Management of data, pages 551–562, 2004. [18] R. S. Roy. 3.9 conversational question answering over knowledge graphs. Conversational Search, page 45. [19] H. Su, X. Shen, R. Zhang, F. Sun, P. Hu, C. Niu, and J. Zhou. Improving multi-turn dialogue modelling with utterance rewriter. arXiv preprint arXiv:1906.07004, 2019. [20] S. Vakulenko, S. Longpre, Z. Tu, and R. Anantha. Question rewriting for conversa- tional question answering. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pages 355–363, 2021. [21] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [22] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault,R. Louf, M. Funtowicz, et al. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019. [23] S.Yu,Z.Liu,C.Xiong,T.Feng,andZ.Liu.Few-shotconversationaldenseretrieval. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 829–838, 2021. [24] Y. Zhang, Z. Li, J. Wang, N. Cheng, and J. Xiao. Self-attention for incomplete utter- ance rewriting. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8047–8051. IEEE, 2022. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/87268 | - |
| dc.description.abstract | 智慧助理越來越普及的環境下，人類與機器的對話互動更趨近人與人之間的口說關係，因此，人類的對話習慣中，省略主詞或使用代名詞使得語意不完整的現象，對機器理解是一項重大的挑戰。機器必須從歷史紀錄中抽取對話中被省略的資訊，將使用者的問題重新組織成完整的問句，再依完整問句來搜尋回答。
本篇論文分析不同模型在對話式問答的問題重寫上的表現，以及在不同的資料集和不同資料特性上的表現與泛化能力等差異。並且提出一種在訓練過程中，利用知識蒸餾的技術加強模型的能力的方法，以提高改寫後的問題品質。 | zh_TW |
| dc.description.abstract | The dialogue between humans and machines is more similar to the oral language with the increasing popularity of intelligent assistants. Therefore, the task of incomplete utterance rewriting (IUR) in multi-turns conversations is a major challenge for machines to understand. To answer user's questions, the machines will extract the omitted information form historical records , reconstruct user's utterances into complete questions and response to the questions. This paper analyzes the performance of different models on incomplete utterance rewriting tasks and compare the differences on generalization ability, data features and data domains. In addition, we propose a knowledge distillation techniques to strengthen the performance of the models, and improve the performance of rewritten questions. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2023-05-18T16:44:22Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2023-05-18T16:44:23Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | 致謝 i
摘要 iii Abstract v Contents vii List of Figures xi List of Tables xiii Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Main Contribution 2 1.3 Thesis Structure 3 Chapter 2 Background 5 2.1 DeepLearning Models 5 2.1.1 Transformer 5 2.1.2 GPT2 6 2.1.3 BERT 7 2.2 Question Answering 8 2.2.1 Question Answering 8 2.2.2 ConversationalQuestionAnswering 9 Chapter 3 Related Work 11 3.1 Approaches to Conversational Question Answering 11 3.1.1 Pipeline Approach 11 3.1.2 End-to-End Approach 12 3.2 Incomplete Utterance Rewriting 13 Chapter 4 Preliminary Study 15 4.1 Datasets 15 4.2 Input Format and Sample Strategy 16 4.3 Analysis of Generalization Capability 18 4.4 Analysis of Question Features 18 Chapter 5 Proposed Method 21 5.1 Overview 21 5.2 Pre-training Stage 22 5.2.1 Teacher Model Training 22 5.2.2 Extract Ratio and Region 23 5.3 Knowledge Distillation 23 5.3.1 Encoder Knowledge Distillation Loss 24 5.3.2 Decoder Knowledge Distillation Loss 25 Chapter 6 Experiments 27 6.1 Datasets 27 6.2 Baseline Models 27 6.3Evaluation Metrics 28 6.4 Training Details 28 6.4.1 Hyperparameters 30 6.5 MainResults 30 Chapter 7 Discussion 33 7.1 Hyperparameter Grid Search 33 7.2 Ablation Study 34 7.3 Analysis of QA and Retrieval Results 35 7.4 Case Study 36 Chapter 8 Conclusion 39 References 41 | - |
| dc.language.iso | en | - |
| dc.subject | 不完整語句改寫 | zh_TW |
| dc.subject | 對話式問答 | zh_TW |
| dc.subject | 知識蒸餾 | zh_TW |
| dc.subject | Conversational Question Answering | en |
| dc.subject | Incomplete Utterance Rewriting | en |
| dc.subject | Knowledge Distillation | en |
| dc.title | 利用知識蒸餾於前後文相關之問題改寫以提升對話式問答 | zh_TW |
| dc.title | Contextual Question Rewriting with Knowledge Distillation for Improving Conversational Question Answering | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 111-1 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 曹昱;李宏毅;蔡宗翰 | zh_TW |
| dc.contributor.oralexamcommittee | Yu Tsao;Hung-yi Lee;Tzong-Han Tsai | en |
| dc.subject.keyword | 對話式問答,不完整語句改寫,知識蒸餾, | zh_TW |
| dc.subject.keyword | Conversational Question Answering,Incomplete Utterance Rewriting,Knowledge Distillation, | en |
| dc.relation.page | 44 | - |
| dc.identifier.doi | 10.6342/NTU202300619 | - |
| dc.rights.note | 同意授權(限校園內公開) | - |
| dc.date.accepted | 2023-02-19 | - |
| dc.contributor.author-college | 電機資訊學院 | - |
| dc.contributor.author-dept | 資訊工程學系 | - |
| Appears in Collections: | 資訊工程學系 | |
Files in This Item:
| File | Size | Format | |
|---|---|---|---|
| ntu-111-1.pdf Access limited in NTU ip range | 561.99 kB | Adobe PDF |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.