請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/67029
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 黃鐘揚(Chung-Yang Huang) | |
dc.contributor.author | Chien-Fu Lin | en |
dc.contributor.author | 林建甫 | zh_TW |
dc.date.accessioned | 2021-06-17T01:17:44Z | - |
dc.date.available | 2017-09-23 | |
dc.date.copyright | 2017-08-24 | |
dc.date.issued | 2017 | |
dc.date.submitted | 2017-08-14 | |
dc.identifier.citation | [1] Serban, Iulian Vlad, et al. 'Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models.' AAAI. 2016.
[2] Williams, Ronald J., and David Zipser. 'A learning algorithm for continually running fully recurrent neural networks.' Neural computation 1.2 (1989): 270-280. [3] Bengio, Samy, et al. 'Scheduled sampling for sequence prediction with recurrent neural networks.' Advances in Neural Information Processing Systems. 2015. [4] Press, Ofir, and Lior Wolf. 'Using the output embedding to improve language models.' arXiv preprint arXiv:1608.05859 (2016). [5] Cho, Kyunghyun, et al. 'Learning phrase representations using RNN encoder-decoder for statistical machine translation.' arXiv preprint arXiv:1406.1078 (2014). [6] Lemon, Oliver, et al. 'An ISU dialogue system exhibiting reinforcement learning of dialogue policies: generic slot-filling in the TALK in-car system.' Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Posters & Demonstrations. Association for Computational Linguistics, 2006. [7] Wang, Zhuoran, and Oliver Lemon. 'A Simple and Generic Belief Tracking Mechanism for the Dialog State Tracking Challenge: On the believability of observed information.' SIGDIAL Conference. 2013. [8] Ritter, Alan, Colin Cherry, and Bill Dolan. 'Unsupervised modeling of twitter conversations.' Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 2010. [9] Shang, Lifeng, Zhengdong Lu, and Hang Li. 'Neural responding machine for short-text conversation.' arXiv preprint arXiv:1503.02364 (2015). [10] Yu, Lantao, et al. 'SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient.' AAAI. 2017. [11] Sutskever, Ilya, Oriol Vinyals, and Quoc V. Le. 'Sequence to sequence learning with neural networks.' Advances in neural information processing systems. 2014. [12] Sordoni, Alessandro, et al. 'A hierarchical recurrent encoder-decoder for generative context-aware query suggestion.' Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 2015. [13] Sun, J. '‘Jieba’Chinese word segmentation tool.' (2012). [14] Rajpurkar, Pranav, et al. 'Squad: 100,000+ questions for machine comprehension of text.' arXiv preprint arXiv:1606.05250 (2016). [15] Li, Jiwei, Minh-Thang Luong, and Dan Jurafsky. 'A hierarchical neural autoencoder for paragraphs and documents.' arXiv preprint arXiv:1506.01057 (2015). [16] Mikolov, Tomas, et al. 'Distributed representations of words and phrases and their compositionality.' Advances in neural information processing systems. 2013. [17] Pennington, Jeffrey, Richard Socher, and Christopher D. Manning. 'Glove: Global vectors for word representation.' EMNLP. Vol. 14. 2014. [18] Le, Quoc, and Tomas Mikolov. 'Distributed representations of sentences and documents.' Proceedings of the 31st International Conference on Machine Learning (ICML-14). 2014. [19] Kiros, Ryan, et al. 'Skip-thought vectors.' Advances in neural information processing systems. 2015. [20] Kingma, Diederik, and Jimmy Ba. 'Adam: A method for stochastic optimization.' arXiv preprint arXiv:1412.6980 (2014). [21] Li, Jiwei, et al. 'Deep reinforcement learning for dialogue generation.' arXiv preprint arXiv:1606.01541 (2016). [22] Rumelhart, David E., Geoffrey E. Hinton, and Ronald J. Williams. 'Learning representations by back-propagating errors.' Cognitive modeling 5.3 (1988): 1. [23] pytorch, Tensors and Dynamic neural networks in Python with strong GPU acceleration. https://github.com/pytorch/pytorch [24] Papineni, Kishore, et al. 'BLEU: a method for automatic evaluation of machine translation.' Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, 2002. [25] Chung, Junyoung, et al. 'Empirical evaluation of gated recurrent neural networks on sequence modeling.' arXiv preprint arXiv:1412.3555 (2014). [26] Srivastava, Nitish, et al. 'Dropout: a simple way to prevent neural networks from overfitting.' Journal of Machine Learning Research 15.1 (2014): 1929-1958. [27] Karpathy, Andrej. 'Char-RNN: Multi-layer recurrent neural networks (LSTM, GRU, RNN) for character-level language models in torch.' (2015). | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/67029 | - |
dc.description.abstract | 在對話生成的領域中,根據應用情境,將對話生成分為任務導向以及開放領域。在任務導向的對話生成模型,由於應用領域上的限制,通常使用規則導向的模型。然而,規則導向的模型具有可拓展性較差和應用範疇受到手刻特徵限制的缺點。
本研究採用端到端神經網絡模型,而非規則導向的模型,以確保模型的一般性。我們採用階層式遞歸編碼器解碼器(HRED)架構作為我們的模型,並擴展了此架構,以便能夠動態地控制生成對話的長度。我們證明由任務導向的語料庫訓練的端到端神經網絡可以產生特定領域的對話。 此外,我們還探討了HRED將對話向量化的潛力。透過HRED生成的嵌入式向量在我們的實驗中勝過其他嵌入方法。 | zh_TW |
dc.description.abstract | In the domain of dialogue generating, there are two types of tasks according to application targets: task-oriented task and open-domain task. Limited by the specific usages, the task-oriented application usually adopts rule-based models. However, rule-based models are hard to extend and the applicable domains are constricted by the hand-craft features.
Instead of using the rule-based models for the task-oriented target, we apply an end-to-end neural network model trained by the task-oriented corpus to ensure the generalization of our model. We adopt the hierarchical recurrent encoder-decoder (HRED) architecture as our model and extend it to dynamically control the generated dialog’s length. This work demonstrates that the end-to-end neural network trained by task-oriented corpus could generate specific domain dialogue. In addition, we explore the potential of the HRED on embedding the dialogue. The embedded vectors created by HRED outperforms other embedding methods on our experiment. | en |
dc.description.provenance | Made available in DSpace on 2021-06-17T01:17:44Z (GMT). No. of bitstreams: 1 ntu-106-R04943092-1.pdf: 1945879 bytes, checksum: 40311d02fa40a8d618c03d071de91db2 (MD5) Previous issue date: 2017 | en |
dc.description.tableofcontents | 口試委員審定書 i
誌謝 ii 中文摘要 iii ABSTRACT iv CONTENTS v LIST OF FIGURES vii LIST OF TABLES ix Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Related Researches 3 1.3 Thesis Organization 4 Chapter 2 Preliminaries 6 2.1 Trie Tree 6 2.2 Directed Acyclic Graph 7 2.3 Neural Network 7 2.3.1 Recurrent Neural Network 9 2.3.2 Sequence to Sequence Model 12 Chapter 3 Chinese Data Processing 13 3.1 Tokenization 13 3.2 Probability Expression of Chinese Tokenization 14 Chapter 4 Generative Model 19 4.1 Dialogue Model 19 4.2 Probability Model 25 4.3 Extended Hierarchical Recursive Neural Network 26 4.3.1 Model Architecture 27 4.3.2 Word Embedding 35 4.3.3 Training 42 4.3.4 Inferencing 48 Chapter 5 Experiments 53 5.1 ASUS Dataset Analysis 53 5.2 Empirical Results 57 5.2.1 Experimental Setting 58 5.2.2 Evaluation Metrics 59 5.2.3 Generated Dialogues 64 5.2.4 Dialogue Embedding 74 Chapter 6 Conclusion 79 REFERENCE 82 | |
dc.language.iso | en | |
dc.title | 利用動態階層式遞歸編碼器解碼器架構生成繁體中文對話 | zh_TW |
dc.title | Using Dynamic Hierarchical Recurrent Encoder-Decoder Architecture to Generate Traditional Chinese Dialogue | en |
dc.type | Thesis | |
dc.date.schoolyear | 105-2 | |
dc.description.degree | 碩士 | |
dc.contributor.oralexamcommittee | 許萬寶,周俊男,李宏毅(Hung-Yi Lee) | |
dc.subject.keyword | 對話生成,階層式遞歸編碼器解碼器,深度學習,端到端模型,繁體中文文本, | zh_TW |
dc.subject.keyword | dialogue generating,hierarchical recurrent encoder-decoder,deep learning,end-to-end model,Traditional Chinese corpus, | en |
dc.relation.page | 85 | |
dc.identifier.doi | 10.6342/NTU201703159 | |
dc.rights.note | 有償授權 | |
dc.date.accepted | 2017-08-14 | |
dc.contributor.author-college | 電機資訊學院 | zh_TW |
dc.contributor.author-dept | 電子工程學研究所 | zh_TW |
顯示於系所單位: | 電子工程學研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-106-1.pdf 目前未授權公開取用 | 1.9 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。