請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/87595
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 傅立成 | zh_TW |
dc.contributor.advisor | Li-Chen Fu | en |
dc.contributor.author | 宋豐裕 | zh_TW |
dc.contributor.author | Feng-Yu Sung | en |
dc.date.accessioned | 2023-06-20T16:15:41Z | - |
dc.date.available | 2023-11-09 | - |
dc.date.copyright | 2023-06-20 | - |
dc.date.issued | 2023 | - |
dc.date.submitted | 2023-02-14 | - |
dc.identifier.citation | [1] W. Yang, Y. Xie, A. Lin, X. Li, L. Tan, K. Xiong, M. Li, and J. Lin, "End-to-End Open-Domain Question Answering with BERTserini," in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 72-77.
[2] P. Yang, H. Fang, and J. Lin, "Anserini: Enabling the use of lucene for information retrieval research," in Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, 2017, pp. 1253-1256. [3] P. Yang, H. Fang, and J. Lin, "Anserini: Reproducible ranking baselines using Lucene," Journal of Data and Information Quality (JDIQ), vol. 10, no. 4, pp. 1-20, 2018. [4] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019, pp. 4171-4186. [5] V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, "Dense Passage Retrieval for Open-Domain Question Answering," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 6769-6781. [6] T. Zhao, X. Lu, and K. Lee, "SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval," in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 565-575. [7] Y. Xie, W. Yang, L. Tan, K. Xiong, N. J. Yuan, B. Huai, M. Li, and J. Lin, "Distant supervision for multi-stage fine-tuning in retrieval-based question answering," in Proceedings of The Web Conference 2020, 2020, pp. 2934-2940. [8] K. Lee, M.-W. Chang, and K. Toutanova, "Latent Retrieval for Weakly Supervised Open Domain Question Answering," in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 6086-6096. [9] M. Iwamura, K. Kunze, Y. Kato, Y. Utsumi, and K. Kise, "Haven't we met before? A realistic memory assistance system to remind you of the person in front of you," in Proceedings of the 5th augmented human international conference, 2014, pp. 1-4. [10] C. Bermejo, T. Braud, J. Yang, S. Mirjafari, B. Shi, Y. Xiao, and P. Hui, "VIMES: A wearable memory assistance system for automatic information retrieval," in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 3191-3200. [11] C. Y. Yang, E. Gamborino, L. C. Fu, and Y. L. Chang, "A Brain-Inspired Self-Organizing Episodic Memory Model for a Memory Assistance Robot," IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 2, pp. 617-628, 2022, doi: 10.1109/TCDS.2021.3061659. [12] E. Tulving, "Episodic and semantic memory," in Organization of memory. Oxford, England: Academic Press, 1972, pp. xiii, 423-xiii, 423. [13] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017. [14] M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, S. Cornell, L. Lugosch, C. Subakan, N. Dawalatabad, A. Heba, and J. Zhong, "SpeechBrain: A general-purpose speech toolkit," arXiv preprint arXiv:2106.04624, 2021. [15] N. Reimers and I. Gurevych, "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3982-3992. [16] Y. Cui, W. Che, T. Liu, B. Qin, S. Wang, and G. Hu, "Revisiting Pre-Trained Models for Chinese Natural Language Processing," in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 657-668. [17] M. Henderson, R. Al-Rfou, B. Strope, Y.-H. Sung, L. Lukács, R. Guo, S. Kumar, B. Miklos, and R. Kurzweil, "Efficient natural language response suggestion for smart reply," arXiv preprint arXiv:1705.00652, 2017. [18] S. Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, and M. Gatford, "Okapi at TREC-3," presented at the Overview of the Third Text REtrieval Conference (TREC-3), January, 1995. [Online]. Available: https://www.microsoft.com/en-us/research/publication/okapi-at-trec-3/. [19] S. Robertson and H. Zaragoza, "The probabilistic relevance framework: BM25 and beyond," Foundations and Trends® in Information Retrieval, vol. 3, no. 4, pp. 333-389, 2009. [20] Y. Qu, Y. Ding, J. Liu, K. Liu, R. Ren, W. X. Zhao, D. Dong, H. Wu, and H. Wang, "RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering," in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 5835-5847. [21] R. Ren, Y. Qu, J. Liu, W. X. Zhao, Q. She, H. Wu, H. Wang, and J.-R. Wen, "RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking," in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 2825-2835. [22] N. Thakur, N. Reimers, J. Daxenberger, and I. Gurevych, "Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks," in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 296-310. [23] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, and L. Antiga, "Pytorch: An imperative style, high-performance deep learning library," Advances in neural information processing systems, vol. 32, 2019. [24] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, and M. Funtowicz, "Transformers: State-of-the-art natural language processing," in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38-45. [25] S. Iyer, S. Min, Y. Mehdad, and W.-t. Yih, "RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering," in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 1280-1287. [26] A.-Z. Yen, H.-H. Huang, and H.-H. Chen, "Personal knowledge base construction from text-based lifelogs," in Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019, pp. 185-194. [27] 黃居仁 (2007-2009) 、謝舒凱 (2009-2010) :跨語言知識表徵基礎架構─面向多語化與全球化的語言學研究。國科會專題補助計畫 (NSC 96-2411-H-003-061-MY3) [28] Chu-Ren Huang and Shu-Kai Hsieh. (2010). Infrastructure for Cross-lingual Knowledge Representation ─ Towards Multilingualism in Linguistic Studies. Taiwan NSC-granted Research Project (NSC 96-2411-H-003-061-MY3) [29] 黃居仁, 謝舒凱, 洪嘉馡, 陳韻竹, 蘇依莉, 陳永祥, 黃勝偉. 中文詞彙網路:跨語言知識處理基礎架構的設計理念與實踐. 中國語文,24卷第二期 [30] T.-H. Yang, H.-H. Huang, A.-Z. Yen, and H.-H. Chen, "Transfer of Frames from English FrameNet to Construct Chinese FrameNet: A Bilingual Corpus-Based Approach," in LREC, 2018. [31] M.-c. Liu and T.-y. Chiang, "The construction of Mandarin VerbNet: A frame-based study of statement verbs," Language and Linguistics, vol. 9, no. 2, pp. 239-270, 2008. [32] H. He and J. D. Choi, "The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders," in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 5555-5577. [33] Y. Cui, T. Liu, W. Che, L. Xiao, Z. Chen, W. Ma, S. Wang, and G. Hu, "A Span-Extraction Dataset for Chinese Machine Reading Comprehension," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 5883-5889. [34] C. C. Shao, T. Liu, Y. Lai, Y. Tseng, and S. Tsai, "Drcd: a chinese machine reading comprehension dataset," arXiv preprint arXiv:1806.00920, 2018. [35] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, "SQuAD: 100,000+ Questions for Machine Comprehension of Text," in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016, pp. 2383-2392. [36] P. Rajpurkar, R. Jia, and P. Liang, "Know What You Don’t Know: Unanswerable Questions for SQuAD," in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2018, pp. 784-789. [37] J. Lin, X. Ma, S.-C. Lin, J.-H. Yang, R. Pradeep, and R. Nogueira, "Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations," in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021, pp. 2356-2362. [38] I. Loshchilov and F. Hutter, "Decoupled weight decay regularization," arXiv preprint arXiv:1711.05101, 2017. [39] J. Brooke, "SUS-A quick and dirty usability scale," Usability evaluation in industry, vol. 189, no. 194, pp. 4-7, 1996. [40] J. Sauro. "Measuring Usability with the System Usability Scale (SUS)." https://measuringu.com/sus/ [41] J. Liu, L. Min, and X. Huang, "An overview of event extraction and its applications," arXiv preprint arXiv:2111.03212, 2021. [42] X. Du and C. Cardie, "Event Extraction by Answering (Almost) Natural Questions," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 671-683. [43] Y.-T. Hsiao, E. Gamborino, and L.-C. Fu, "A Hybrid Conversational Agent with Semantic Association of Autobiographic Memories for the Elderly," in Cross-Cultural Design. Applications in Health, Learning, Communication, and Creativity: 12th International Conference, CCD 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22, 2020: Springer, pp. 53-66. [44] D. S. Batista. "Named-Entity evaluation metrics based on entity-level." https://www.davidsbatista.net/blog/2018/05/09/Named_Entity_Evaluation/ | - |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/87595 | - |
dc.description.abstract | 在日常生活,人們需要記憶各式各樣的事情,也時常需要作筆記,避免日後遺忘。因此,本論文提出一個基於記憶庫問答的記憶輔助機器人,主要以語音作為輸入,幫助使用者儲存記憶和回憶。
我們所提出系統有多個模組,包含記憶儲存模組、記憶庫問答模組和記憶日曆模組。記憶儲存模組能讓使用者透過語音來儲存記憶,我們的系統會通過語音辨識來得到逐字稿,接著使用多種方法抽取特徵,建立記憶並存儲在記憶庫中。而記憶庫問答模組,目的是讓使用者透過問答的方式回憶。根據問題,我們基於開放領域問答的架構來進行檢索與預測,接著使用一個答案排序的方法來得到最終的答案。除此之外,為了幫助使用者瀏覽記憶,我們提出一個記憶日曆模組,從逐字稿中提取日常事件,並顯示在日曆上。實驗部分,我們使用兩個實驗進行評估,第一個實驗為開放領域問答的實驗,而第二個實驗則是人機互動實驗。 | zh_TW |
dc.description.abstract | In daily life, people need to memorize various things and often need to write notes to avoid forgetting in the future. Therefore, this thesis proposes a memory assistance robot based on memory bank question answering, which mainly uses speech as input to help the user store and recall memories.
Our proposed system contains multiple modules, including memory storage module, memory bank question answering module, and memory calendar module. The memory storage module allows users to store memories through speech. Our system will obtain transcript through speech recognition and then use various methods to extract features, construct the memory, and store it in the memory bank. For the memory bank question answering module, the purpose is to allow the user to recall through question answering. According to the query, we perform retrieval and prediction based on the open-domain question answering architecture. Then, we use an answer ranking method to get the final answer. In addition, to help the user view memories, we propose a memory calendar module, which extracts daily events from the transcript and shows them on the calendar. In the experiment part, we use two experiments for evaluation. The first experiment is the open-domain question answering experiment, and the second experiment is the human-robot interaction experiment. | en |
dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2023-06-20T16:15:41Z No. of bitstreams: 0 | en |
dc.description.provenance | Made available in DSpace on 2023-06-20T16:15:41Z (GMT). No. of bitstreams: 0 | en |
dc.description.tableofcontents | 口試委員會審定書 i
誌謝 ii 中文摘要 iii ABSTRACT iv CONTENTS v LIST OF FIGURES viii LIST OF TABLES x Chapter 1 Introduction 1 1.1 Background 1 1.2 Motivation 1 1.3 Related Work 2 1.3.1 Open-Domain Question Answering 2 1.3.2 Memory Assistance System 3 1.3.3 Comparison 3 1.4 Objectives and Contributions 4 1.5 Thesis Organization 6 Chapter 2 Preliminaries 7 2.1 Human Memory 7 2.1.1 Episodic Memory and Semantic Memory 7 2.1.2 Retrospective memory and Prospective memory 7 2.2 BERT 8 2.3 Natural Language Processing Tasks 8 2.4 Bi-encoder and Cross-encoder 9 Chapter 3 Memory Assistance Robot 10 3.1 System Overview 10 3.2 Speech Recognition 12 3.3 Memory Feature Extraction 13 3.4 Memory Bank and Elasticsearch 18 3.5 Query Feature Extraction 20 3.6 Memory Feature Matching 22 3.6.1 Entity Matching 24 3.6.2 BM25 25 3.6.3 BM25 with Entity Filtering 27 3.6.4 Dense Retrieval 28 3.6.5 Dense Retrieval with Entity Filtering 29 3.6.6 Postprocessing 29 3.7 Memory Ranking 30 3.8 Answer Prediction and Ranking 31 3.8.1 Answer Prediction 32 3.8.2 Answer-focused verification 35 3.8.3 Answer Ranking 40 3.9 Memory Calendar 41 3.9.1 Modified Word Sense Disambiguation 41 3.9.2 Daily Event Extraction 47 3.9.3 Event Filtering 51 3.9.4 Google Calendar 54 3.10 Robot Interface and Backend Server 54 3.10.1 Robot Interface 55 3.10.2 Backend Server 55 Chapter 4 Experiments 57 4.1 Open-Domain QA Experiment 57 4.1.1 Open-domain QA dataset 57 4.1.2 Chinese Wikipedia 59 4.1.3 Chinese Wikipedia setup 60 4.1.4 Implementation of models 61 4.1.5 System and parameter settings 69 4.1.6 Evaluation metrics 71 4.1.7 Experimental results 72 4.1.8 Ablation study 73 4.2 Human-Robot Interaction Experiment 76 4.2.1 Participants and Environments 76 4.2.2 Template story and questions 77 4.2.3 Designed Stories 78 4.2.4 Personal Story 79 4.2.5 Questions 80 4.2.6 Evaluation methods 83 4.2.7 Experiment settings 89 4.2.8 Procedure 92 4.2.9 Experimental results 92 4.2.10 Discussion 99 Chapter 5 Conclusions 102 REFERENCES 104 | - |
dc.language.iso | en | - |
dc.title | 基於記憶庫問答的記憶輔助機器人 | zh_TW |
dc.title | Memory Assistance Robot based on Memory Bank Question Answering | en |
dc.type | Thesis | - |
dc.date.schoolyear | 111-1 | - |
dc.description.degree | 碩士 | - |
dc.contributor.oralexamcommittee | 林守德;邱銘章;李宏毅;張玉玲 | zh_TW |
dc.contributor.oralexamcommittee | Shou-De Lin;Ming-Jang Chiu;Hung-Yi Lee;Yu-Ling Chang | en |
dc.subject.keyword | 記憶輔助機器人,記憶輔助,開放領域問答,問答系統, | zh_TW |
dc.subject.keyword | Memory Assistance Robot,Memory Assistance,Open-Domain Question Answering,Question Answering, | en |
dc.relation.page | 108 | - |
dc.identifier.doi | 10.6342/NTU202300222 | - |
dc.rights.note | 未授權 | - |
dc.date.accepted | 2023-02-15 | - |
dc.contributor.author-college | 電機資訊學院 | - |
dc.contributor.author-dept | 資訊網路與多媒體研究所 | - |
顯示於系所單位: | 資訊網路與多媒體研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-111-1.pdf 目前未授權公開取用 | 5.79 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。