請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/73839
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 魏志平(Chih-Ping Wei) | |
dc.contributor.author | Kuan-Ting Lai | en |
dc.contributor.author | 賴冠廷 | zh_TW |
dc.date.accessioned | 2021-06-17T08:11:34Z | - |
dc.date.available | 2021-08-19 | |
dc.date.copyright | 2019-08-19 | |
dc.date.issued | 2019 | |
dc.date.submitted | 2019-08-15 | |
dc.identifier.citation | Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. cite arxiv:1409.0473Comment: Accepted at ICLR 2015 as oral presentation.
Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N. (2015). Scheduled sampling for se- quence prediction with recurrent neural networks. In Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., and Garnett, R., editors, Advances in Neural Information Processing Systems 28, pages 1171–1179. Curran Associates, Inc. Che, T., Li, Y., Zhang, R., Devon Hjelm, R., Li, W., Song, Y., and Bengio, Y. (2017). Maximum-Likelihood Augmented Discrete Generative Adversarial Networks. arXiv e-prints, page arXiv:1702.07983. Chen, D., Bolton, J., and Manning, C. D. (2016). A thorough examination of the CNN/daily mail reading comprehension task. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2358–2367, Berlin, Germany. Association for Computational Linguistics. Cho, K., van Merrie¨nboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734, Doha, Qatar. Association for Computational Linguistics. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. cite arxiv:1412.3555Comment: Pre- sented in NIPS 2014 Deep Learning and Representation Learning Workshop. Clark, C. and Gardner, M. (2018). Simple and effective multi-paragraph reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 845–855, Melbourne, Aus- tralia. Association for Computational Linguistics. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. Duan, N., Tang, D., Chen, P., and Zhou, M. (2017). Question generation for question answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 866–874, Copenhagen, Denmark. Association for Computational Linguistics. Fellbaum, C., editor (1998). WordNet: an electronic lexical database. MIT Press. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D., and Weinberger, K. Q., editors, Advances in Neural Information Processing Systems 27, pages 2672–2680. Curran Associates, Inc. Gu, J., Lu, Z., Li, H., and Li, V. O. (2016). Incorporating copying mechanism in sequence- to-sequence learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1631–1640, Berlin, Germany. Association for Computational Linguistics. He, W., Liu, K., Liu, J., Lyu, Y., Zhao, S., Xiao, X., Liu, Y., Wang, Y., Wu, H., She, Q., Liu, X., Wu, T., and Wang, H. (2018). DuReader: a Chinese machine reading com- prehension dataset from real-world applications. In Proceedings of the Workshop on Machine Reading for Question Answering, pages 37–46, Melbourne, Australia. Association for Computational Linguistics. Hu, M., Wei, F., Peng, Y., Huang, Z., Yang, N., and Li, D. (2018). Read + Verify: Machine Reading Comprehension with Unanswerable Questions. arXiv e-prints, page arXiv:1808.05759. Husza´r, F. (2015). How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary? arXiv e-prints, page arXiv:1511.05101. Jia, R. and Liang, P. (2017). Adversarial examples for evaluating reading comprehen- sion systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021–2031, Copenhagen, Denmark. Association for Com- putational Linguistics. Kim, Y., Lee, H., Shin, J., and Jung, K. (2019). Improving neural question generation us- ing answer separation. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):6602–6609. Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. cite arxiv:1412.6980Comment: Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015. Lai, G., Xie, Q., Liu, H., Yang, Y., and Hovy, E. (2017). RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785–794, Copenhagen, Denmark. Association for Computational Linguistics. Levy, O., Seo, M., Choi, E., and Zettlemoyer, L. (2017). Zero-shot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 333–342, Vancouver, Canada. As- sociation for Computational Linguistics. Luong, T., Pham, H., and Manning, C. D. (2015). Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, Lisbon, Portugal. Asso- ciation for Computational Linguistics. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv e-prints, page arXiv:1301.3781. Mirza, M. and Osindero, S. (2014). Conditional Generative Adversarial Nets. arXiv e-prints, page arXiv:1411.1784. Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., and Deng, L. (2016). MS MARCO: A human generated machine reading comprehension dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, December 9, 2016. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). Bleu: A method for auto- matic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318, Stroudsburg, PA, USA. Association for Computational Linguistics. Pennington, J., Socher, R., and Manning, C. (2014). Glove: Global vectors for word rep- resentation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227– 2237, New Orleans, Louisiana. Association for Computational Linguistics. Radford, A., Narasimhan, K., Rockta¨schel, T., and Sutskever, I. (2018). Improving Lan- guage Understanding by Generative Pre-Training. OpenAI, page 12. Rajpurkar, P., Jia, R., and Liang, P. (2018). Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789, Melbourne, Australia. Association for Computational Linguistics. Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. (2016). SQuAD: 100,000+ ques- tions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. Seo, M., Kembhavi, A., Farhadi, A., and Hajishirzi, H. (2017). Bidirectional at- tention flow for machine comprehension. In International Conference on Learning Representations. Shen, Y., Huang, P.-S., Gao, J., and Chen, W. (2017). Reasonet: Learning to stop reading in machine comprehension. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, pages 1047–1055, New York, NY, USA. ACM. Sun, F., Li, L., Qiu, X., and Liu, Y. (2018). U-Net: Machine Reading Comprehension with Unanswerable Questions. arXiv e-prints, page arXiv:1810.06638. Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D., and Weinberger, K. Q., editors, Advances in Neural Information Processing Systems 27, pages 3104–3112. Curran Associates, Inc. Sutton, R. S., McAllester, D. A., Singh, S. P., and Mansour, Y. (2000). Policy gradient methods for reinforcement learning with function approximation. In Solla, S. A., Leen, T. K., and Mu¨ller, K., editors, Advances in Neural Information Processing Systems 12, pages 1057–1063. MIT Press. Tan, C., Wei, F., Zhou, Q., Yang, N., Lv, W., and Zhou, M. (2018). I know there is no answer: Modeling answer validation for machine reading comprehension. In Zhang, M., Ng, V., Zhao, D., Li, S., and Zan, H., editors, Natural Language Processing and Chinese Computing, pages 85–97, Cham. Springer International Publishing. Tang, D., Duan, N., Qin, T., Yan, Z., and Zhou, M. (2017). Question Answering and Question Generation as Dual Tasks. arXiv e-prints, page arXiv:1706.02027. Tang, D., Duan, N., Yan, Z., Zhang, Z., Sun, Y., Liu, S., Lv, Y., and Zhou, M. (2018). Learning to collaborate for question answering and asking. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1564– 1574, New Orleans, Louisiana. Association for Computational Linguistics. Taylor, W. L. (1953). “cloze procedure”: A new tool for measuring readability. Journalism Bulletin, 30(4):415–433. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. (2017). Attention is all you need. In Guyon, I., Luxburg, U. V., Ben- gio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc. Wang, W., Yang, N., Wei, F., Chang, B., and Zhou, M. (2017). Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers), volume 1, pages 189–198. Weissenborn, D., Wiese, G., and Seiffe, L. (2017). Making neural QA as simple as possi- ble but not simpler. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 271–280, Vancouver, Canada. Association for Computational Linguistics. Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3):229–256. Yu, L., Zhang, W., Wang, J., and Yu, Y. (2017). Seqgan: Sequence generative adversarial nets with policy gradient. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, pages 2852–2858. Association for the Advancement of Artificial Intelligence. Zhou, Q., Yang, N., Wei, F., Tan, C., Bao, H., and Zhou, M. (2018). Neural question generation from text: A preliminary study. In Huang, X., Jiang, J., Zhao, D., Feng, Y., and Hong, Y., editors, Natural Language Processing and Chinese Computing, pages 662–671, Cham. Springer International Publishing. | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/73839 | - |
dc.description.abstract | 儘管隨著深度學習模型技術的研發以及硬體計算能力的提升讓機器閱讀理解系統獲得快速的發展,在現實的條件中往往有許多模型可靠性的考量,一個缺乏可靠性的閱讀理解模型仍然欠缺實用性。在這篇研究中我們提出兩項閱讀理解任務的可靠性問題: 訓練資料稀少以及缺乏無法回答的問句的訓練資料,前者會造成模型無法充分發揮並有過度擬合的問題;後者則造成模型遇到無法回答的問句時會亂猜答案。
我們利用問句生成的技術希望能夠透過現有的資料產生出更多的問句進而有更多的閱讀理解訓練資料,同時,結合現有資料與生成資料時賦予每一筆資料一個來自問句評斷器的權重,來平衡品質好壞的問句對模型所造成的影響。對於缺少無法回答問句的問題,我們進一步利用對抗式生成網路的結構來進一步調整預訓練好的問句生成器,這樣的技術能夠解決預訓練時最大似然估計法所產生的問題,並產生出更接近真實問句的結果,另外,再將現有的可回答問句透過我們提出的無法回答問句替換規則轉化為一個偽無法回答問句,即可以用來訓練閱讀理解模型讓模型學習一些無法回答問句的辨識方法。 在許多實驗中也證明了我們提出的資料擴增方法是有效的,跟現有方法比較能夠一定程度提升閱讀理解模型的回答可靠性,同時也進一步分析實驗結果並討論我們的方法的優點與限制所在。 | zh_TW |
dc.description.abstract | Despite the popularity of deep learning techniques applied in machine reading comprehension (MRC) systems, the robustness issues of the systems may slow down their deployment in real-world scenarios. We describe two of the robustness issues as data-limited condition MRC, which may constrain the capacity of the resultant model, and MRC without unanswerable questions, which makes unreliable guesses on unanswerable questions. In this research, we exploit the question generation (QG) technique aiming to expand the existing training triplets and loss weighting by a question discriminator to balance the influence of different quality questions. Generative adversarial net is further incorporated into the QG learning to alleviate the exposure bias caused by maximum likelihood estimation training. We also propose unanswerable question perturbation rules that convert an answerable question to a pseudo unanswerable one, which can be used to teach the MRC model what they do not know. Extensive experiments are conducted on these two tasks and demonstrate significant improvements over the baselines. We also analyze the experiment results and discuss the pros and cons of our proposed methods. | en |
dc.description.provenance | Made available in DSpace on 2021-06-17T08:11:34Z (GMT). No. of bitstreams: 1 ntu-108-R06725007-1.pdf: 1758633 bytes, checksum: 0a186655f699a1f5e6f56faa83f8e436 (MD5) Previous issue date: 2019 | en |
dc.description.tableofcontents | 口試委員會審定書i
誌謝ii 摘要iii Abstract iv List of Figures viii List of Tables x Chapter 1 Introduction 1 Chapter 2 Literature Review 5 2.1 Machine Reading Comprehension only with Answerable Questions 5 2.2 Machine Reading Comprehension with Unanswerable Questions 7 2.3 Question Generation 8 2.4 GAN for Natural Language Generation 9 Chapter 3 Data-Limited Machine Reading Comprehension 11 3.1 QG Learning 12 3.1.1 Data Preparation 13 3.1.2 QG Maximum Likelihood Estimation Training 14 3.1.3 QD Training 18 3.1.4 GAN Fine-tuning 20 3.2 Answer Selection and Synthetic Question Generation 22 3.2.1 Answer Selection 23 3.2.2 Synthetic Question Generation 24 3.3 MRC Learning 26 3.3.1 Loss Weighting 26 3.3.2 MRC Model Selection 27 Chapter 4 Empirical Evaluations of Data-Limited MRC 29 4.1 Dataset and Evaluation Metrics 29 4.2 Experiment Settings 30 4.2.1 Implementation Details 30 4.2.2 Benchmark Selection 31 4.2.3 Variants of Our Method 34 4.3 Main Results 34 4.4 Discussions 36 Chapter 5 Machine Reading Comprehension without Unanswerable Questions 39 5.1 Unanswerable Question Perturbation Rules 40 5.2 MRC Learning with Both Answerable and Unanswerable Questions 42 Chapter 6 Empirical Evaluations of Machine Reading Comprehension without Unanswerable Questions 44 6.1 Dataset and Evaluation Metrics 44 6.2 Experiment Settings 45 6.2.1 Benchmark Selection 45 6.2.2 Variants of Our Method 45 6.3 Main Results 46 6.4 Discussions 48 Chapter 7 Conclusion 51 References 53 Appendix 62 A Question Generation Examples 62 | |
dc.language.iso | en | |
dc.title | RRCGAN:對抗式學習強化機器閱讀理解模型可靠性 | zh_TW |
dc.title | RRCGAN: Robust Machine Reading Comprehension with
Adversarial Learning | en |
dc.type | Thesis | |
dc.date.schoolyear | 107-2 | |
dc.description.degree | 碩士 | |
dc.contributor.oralexamcommittee | 簡立峰(Lee-Feng Chien),楊錦生(Chin-Sheng Yang) | |
dc.subject.keyword | 機器閱讀理解,問題生成,對抗式學習,無法回答問句生成規則, | zh_TW |
dc.subject.keyword | machine reading comprehension,question generation,adversarial learning,unanswerable question perturbation rules, | en |
dc.relation.page | 66 | |
dc.identifier.doi | 10.6342/NTU201901920 | |
dc.rights.note | 有償授權 | |
dc.date.accepted | 2019-08-16 | |
dc.contributor.author-college | 管理學院 | zh_TW |
dc.contributor.author-dept | 資訊管理學研究所 | zh_TW |
顯示於系所單位: | 資訊管理學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-108-1.pdf 目前未授權公開取用 | 1.72 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。