請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/8317
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 魏志平(Chih-Ping Wei) | |
dc.contributor.author | Yao-Chang Chu | en |
dc.contributor.author | 朱瑤章 | zh_TW |
dc.date.accessioned | 2021-05-20T00:51:59Z | - |
dc.date.available | 2020-08-07 | |
dc.date.available | 2021-05-20T00:51:59Z | - |
dc.date.copyright | 2020-08-07 | |
dc.date.issued | 2020 | |
dc.date.submitted | 2020-08-05 | |
dc.identifier.citation | Agichtein, E. and Gravano, L. (2000). Snowball: Extracting relations from large plain-text collections. Proceedings of the Fifth ACM Conference on Digital Libraries (DL). Aronson, A. R. and Lang, F.-M. (2010). An overview of Metamap: historical perspective and recent advances. Journal of the American Medical Informatics Association, 17(3):229–236. Asahara, M. and Matsumoto, Y. (2003). Japanese named entity extraction with redundant morphological analysis. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 8–15. Baldini Soares, L., FitzGerald, N., Ling, J., and Kwiatkowski, T. (2019). Matching the blanks: Distributional similarity for relation learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2895–2905, Florence, Italy. Association for Computational Linguistics. Bick, E. (2004). A named entity recognizer for Danish. In Proceedings of the 4th International Conference on Language Resources and Evaluation, pages 305–308. Bikel, D. M., Miller, S., Schwartz, R., and Weischedel, R. (1998). Nymble: a high-performance learning name-finder. arXiv preprint cmp-lg/9803003. Bodenreider, O. (2004). The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Research, 32(suppl 1):D267–D270. Bravo, A., Pinero, J., Queralt-Rosinach, N., Rautschka, M., and Furlong, L. I. (2015). Extraction of relations between genes and diseases from text and large-scale data analysis: implications for translational research. BMC Bioinformatics, 16(1):55. Brin, S. (1999). Extracting patterns and relations from the world wide web. In Atzeni, P., Mendelzon, A., and Mecca, G., editors, The World Wide Web and Databases, pages 172–183, Berlin, Heidelberg. Springer Berlin Heidelberg. Bundschus, M., Dejori, M., Stetter, M., Tresp, V., and Kriegel, H.-P. (2008). Extraction of semantic biomedical relations from text using conditional random fields. BMC Bioinformatics, 9(1):207. Chiu, J. P. and Nichols, E. (2016). Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics, 4:357–370. Collins, M. (2002). Ranking algorithms for named entity extraction: Boosting and the voted perceptron. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 489–496. Dernoncourt, F., Lee, J. Y., and Szolovits, P. (2017). Neuroner: an easy-to-use program for named-entity recognition based on neural networks. arXiv preprint arXiv:1705.05487. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Fu, T.-J., Li, P.-H., and Ma, W.-Y. (2019). Graphrel: Modeling text as relational graphs for joint entity and relation extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1409–1418. Fundel, K., K¨uffner, R., and Zimmer, R. (2007). Relex—relation extraction using dependency parse trees. Bioinformatics, 23(3):365–371. Giorgi, J., Wang, X., Sahar, N., Shin, W. Y., Bader, G. D., and Wang, B. (2019). End-to-end named entity recognition and relation extraction using pre-trained language models. arXiv preprint arXiv:1912.13415. Han, X., Zhu, H., Yu, P., Wang, Z., Yao, Y., Liu, Z., and Sun, M. (2018). Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. arXiv preprint arXiv:1810.10147. Honnibal, M. and Johnson, M. (2015). An improved non-monotonic transition system for dependency parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1373–1378. Ju, M., Miwa, M., and Ananiadou, S. (2018). A neural layered model for nested named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1 (Long Papers), pages 1446–1459. Kambhatla, N. (2004). Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions, pages 22–es. Kavuluru, R., Rios, A., and Tran, T. (2017). Extracting drug-drug interactions with word and character-level recurrent neural networks. In 2017 IEEE International Conference on Healthcare Informatics (ICHI), pages 5–12. IEEE. Kilicoglu, H., Rosemblat, G., Fiszman, M., and Shin, D. (2020). Broad-coverage biomedical relation extraction with SemRep. BMC Bioinformatics, 21:1–28. Kilicoglu, H., Shin, D., Fiszman, M., Rosemblat, G., and Rindflesch, T. C. (2012). SemMedDB: a PubMed-scale repository of biomedical semantic predications. Bioinformatics, 28(23):3158–3160. Kipf, T. N. and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. Kiros, R., Zhu, Y., Salakhutdinov, R. R., Zemel, R., Urtasun, R., Torralba, A., and Fidler, S. (2015). Skip-thought vectors. In Advances in Neural Information Processing Systems, pages 3294–3302. Kolesnikov, A., Zhai, X., and Beyer, L. (2019). Revisiting self-supervised visual representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1920–1929. Krallinger, M., Rabal, O., Akhondi, S. A., Perez, M. P., Santamarıa, J., Rodrıguez, G., et al. (2017). Overview of the biocreative vi chemical-protein interaction track. In Proceedings of the sixth BioCreative Challenge EvaluationWorkshop, volume 1, pages 141–146. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., and Dyer, C. (2016). Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 260–270, San Diego, California. Association for Computational Linguistics. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). Albert: A lite BERT for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., and Kang, J. (2020). Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240. Lin, Y., Shen, S., Liu, Z., Luan, H., and Sun, M. (2016). Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, volume 1 (Long Papers), pages 2124–2133. Lindberg, D. A., Humphreys, B. L., and McCray, A. T. (1993). The unified medical language system. Yearbook of Medical Informatics, 2(01):41–51. Liu, C., Sun, W., Chao, W., and Che, W. (2013). Convolution neural network for relation extraction. In International Conference on Advanced Data Mining and Applications, pages 231–242. Springer. McCray, A. T., Srinivasan, S., and Browne, A. C. (1994). Lexical methods for managing variation in biomedical terminologies. In Proceedings of the Annual Symposium on Computer Application in Medical Care, page 235. American Medical Informatics Association. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111–3119. Mintz, M., Bills, S., Snow, R., and Jurafsky, D. (2009). Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 1003–1011, Suntec, Singapore. Association for Computational Linguistics. Miwa, M. and Bansal, M. (2016). End-to-end relation extraction using LSTMs on sequences and tree structures. arXiv preprint arXiv:1601.00770. Nadeau, D. and Sekine, S. (2007). A survey of named entity recognition and classification. Lingvisticae Investigationes, 30. Neumann, M., King, D., Beltagy, I., and Ammar, W. (2019). Scispacy: Fast and robust models for biomedical natural language processing. arXiv preprint arXiv:1902.07669. Ningthoujam, D., Yadav, S., Bhattacharyya, P., and Ekbal, A. (2019). Relation extraction between the clinical entities based on the shortest dependency path based LSTM. arXiv preprint arXiv:1903.09941. Pawar, S., Palshikar, G. K., and Bhattacharyya, P. (2017). Relation extraction: A survey. arXiv preprint arXiv:1712.05191. Peng, Y., Rios, A., Kavuluru, R., and Lu, Z. (2018). Chemical-protein relation extraction with ensembles of svm, cnn, and rnn models. arXiv preprint arXiv:1802.01255. Pennington, J., Socher, R., and Manning, C. D. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018). Improving language understanding by generative pre-training. Ravichandran, D. and Hovy, E. (2002). Learning surface text patterns for a question answering system. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 41–47, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. Rindflesch, T. C. and Fiszman, M. (2003). The interaction of domain knowledge and linguistic structure in natural language processing: interpreting hypernymic propositions in biomedical text. Journal of Biomedical Informatics, 36(6):462–477. Rindflesch, T. C., Fiszman, M., and Libbus, B. (2005). Semantic interpretation for the biomedical research literature. In Medical Informatics. Integrated Series in Information Systems, volume 8, pages 399–422. Springer. Roth, D. and Yih, W.-t. (2002). Probabilistic reasoning for entity relation recognition. In Proceedings of the 19th International Conference on Computational Linguistics. Roth, D. and Yih,W.-t. (2004). A linear programming formulation for global inference in natural language tasks. Technical report, Department of Computer Science, University of Illinois at Urbana-Champaign. Sekine, S. (1998). Description of the Japanese NE system used for MET-2. In Proceedings of Seventh Message Understanding Conference (MUC-7). Song, L., Zhang, Y., Gildea, D., Yu, M., and Su, Z. W. J. (2019). Leveraging dependency forest for neural medical relation extraction. arXiv preprint arXiv:1911.04123. Srihari, R. and Li, W. (1999). Information extraction supported question answering. Technical report, Cymfony Net Inc., Williamsville, NY. Subasic, P., Yin, H., and Lin, X. (2019). Building knowledge base through deep learning relation extraction and wikidata. In Proceedings of AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering. Sutton, C., McCallum, A., and Rohanimanesh, K. (2007). Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data. Journal of Machine Learning Research, 8(Mar):693–723. Van Mulligen, E. M., Fourrier-Reglat, A., Gurwitz, D., Molokhia, M., Nieto, A., Trifiro, G., Kors, J. A., and Furlong, L. I. (2012). The eu-adr corpus: annotated drugs, diseases, targets, and their relationships. Journal of Biomedical Informatics, 45(5):879–884. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. CoRR, abs/1706.03762. Wei, Z., Su, J., Wang, Y., Tian, Y., and Chang, Y. (2019). A novel hierarchical binary tagging framework for joint extraction of entities and relations. arXiv preprint arXiv:1909.03227. Yao, L., Riedel, S., and McCallum, A. (2010). Collective cross-document relation extraction without labelled data. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1013–1023, Cambridge, MA. Association for Computational Linguistics. Yu, B., Zhang, Z., and Su, J. (2019). Joint extraction of entities and relations based on a novel decomposition strategy. arXiv preprint arXiv:1909.04273. Zeng, D., Liu, K., Chen, Y., and Zhao, J. (2015). Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1753–1762. Zeng, D., Liu, K., Lai, S., Zhou, G., and Zhao, J. (2014). Relation classification via convolutional deep neural network. In Proceedings of the 25th International Conference on Computational Linguistics, pages 2335–2344. Zhang, D. and Wang, D. (2015). Relation classification via recurrent neural network. arXiv preprint arXiv:1508.01006. Zhao, Y., Wan, H., Gao, J., and Lin, Y. (2019). Improving relation classification by entity pair graph. In Proceedings of the Eleventh Asian Conference on Machine Learning, pages 1156–1171. Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., and Xu, B. (2017). Joint extraction of entities and relations based on a novel tagging scheme. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, volume 1 (Long Papers). Zhou, G., Su, J., Zhang, J., and Zhang, M. (2005). Exploring various knowledge in relation extraction. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 427–434. | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/8317 | - |
dc.description.abstract | 關係萃取的任務是從文本中自動學習、抽取兩個實體間的關係。近年來,神經網路模型被廣泛應用在關係萃取上,也取得了優異的表現。然而,神經網路需要大量的訓練資料,而在生醫領域,因為標記成本昂貴,缺乏大量的訓練資料,所以我們進一步探索只需要少量標記資料來微調模型的自監督式學習方法。 MTB 是一個利用自監督式學習方法的關係萃取模型,藉由相同兩實體組成的實體對(entity pair)出現在不同句子也可能隱含相同關係的假設,MTB 得以訓練任意兩實體間的關係向量表示。不像過去許多深度學習之關係萃取模型,MTB 並未利用額外的自然語言特徵,故我們認為若加入兩實體間的依存路徑資訊,有機會讓 MTB 訓練得更好。另外,由於 MTB 僅利用不同的兩實體對是否相同當作訓練依據,負面樣本(非完全相同的實體對)的選定格外重要,因此,我們認為除了 MTB 提出的兩種負面樣本外,還存在使 MTB 訓練更有效的負面樣本。 因此,基於 MTB 模型,我們提出兩個改善方向:(1) 藉由四種網路模組編碼並嵌入實體對之間的依存關係 (2) 藉由行內(inline)負樣本,使 MTB 模型不能只學會關鍵字匹配,而作為真正學到基於上下文的關係表示。在不同設置的實驗下,我們證明了相對於 MTB 原本架構,我們提出的兩個改善方向都能有效地提升關係萃取的效能。我們並探索了在簡單或複雜的句法關係下,更適合的依存神經網路模組,也證明了在更細粒度的方向性關係下,我們的模型仍能有效辨別並超越 MTB 原始架構的表現。 | zh_TW |
dc.description.abstract | Relation extraction is the task that learns and extracts relations between entities from the text. In recent years, neural network models have been widely used in relation extraction, and have achieved the state-of-the-art performance. However, neural networks require a large amount of training data. In the biomedical domain, because acquiring labeled instances is expensive and the training dataset is often small-sized, we further explore self-supervised learning methods that require only a small amount of labeling data for fine-tuning the model. Matching The Blank (MTB) is a self-supervised based relation extraction model. With the assumption that if two entity pair from two sentences are the same, it also implies that they are having the same relation, MTB can train the vector of relation representation between any two entities. However, unlike many deep learning relationship extraction models in the past, MTB does not use additional natural language features other than text. Hence, we believe that if the dependency parsing information between the two entities in a sentence is taken into account, there is an opportunity for MTB to be trained better. In addition, since negative samples play an important role in MTB training, the selection of negative samples (non-identical entity pairs) is particularly important. Therefore, we believe there exists a new type of negative samples that is more effective for MTB training. Therefore, based on the MTB model, we propose two directions for improvement: (1) four neural network modules to encode the dependency relationship between entities, and (2) inline negative samples that the MTB model will not just learn to do keyword matching, but will truly learn context-based relation representation. With the various experiment settings for robustness, we prove that compared with the original structure of MTB, the two directions that we propose can improve the effectiveness of relation extraction. We also explore more suitable dependency modules under simple or complex dependency relationships of an entity pair, and also prove that under more fine-grained directional relations, our model can still effectively identify and outperform the original structure of MTB. | en |
dc.description.provenance | Made available in DSpace on 2021-05-20T00:51:59Z (GMT). No. of bitstreams: 1 U0001-0508202013275700.pdf: 2324478 bytes, checksum: 540b36847dab0dcad0736d273431043d (MD5) Previous issue date: 2020 | en |
dc.description.tableofcontents | 口試委員會審定書 i 誌謝 ii 摘要 iii Abstract iv List of Figures viii List of Tables x Chapter 1 Introduction 1 1.1 Background 1 1.2 Research Motivation 5 1.3 Research Objectives 7 Chapter 2 Literature Review 8 2.1 Pipelined Relation Extraction 8 2.2 Joint learning Relation Extraction 14 2.3 Relation Extraction with few labeled data 16 2.4 Summary 19 Chapter 3 Methodology 20 3.1 Problem Definition of Self-supervised Training 20 3.2 Model Overview 21 3.3 BERT Encoder 22 3.4 Dependency Path Encoder 23 3.5 Merging Layer 29 3.6 Training Process 30 3.7 Few-shot Relation Classification 32 Chapter 4 Empirical Evaluations 33 4.1 Experiment Setting for Self-supervised Learning 33 4.2 Compared Methods 35 4.3 Experiment setting for Few-shot Relation Classification 36 4.4 Experiment Results 39 Chapter 5 Conclusions 46 5.1 Contributions 46 5.2 Future Works 46 References 48 | |
dc.language.iso | en | |
dc.title | 利用依存句法於生物醫學關係萃取之表示學習 | zh_TW |
dc.title | Representation Learning for Biomedical Relation Extraction with Dependency Parsing | en |
dc.type | Thesis | |
dc.date.schoolyear | 108-2 | |
dc.description.degree | 碩士 | |
dc.contributor.oralexamcommittee | 張詠淳(Yung-Chun Chang),吳家齊(Chia-Chi Wu) | |
dc.subject.keyword | 關係萃取,生醫關係萃取,關係分類,深度學習,非監督式學習,自監督式學習, | zh_TW |
dc.subject.keyword | Relation extraction,Biomedical relation extraction,Relation classification,Deep learning,Unsupervised Learning,Self-supervised Learning, | en |
dc.relation.page | 57 | |
dc.identifier.doi | 10.6342/NTU202002454 | |
dc.rights.note | 同意授權(全球公開) | |
dc.date.accepted | 2020-08-05 | |
dc.contributor.author-college | 管理學院 | zh_TW |
dc.contributor.author-dept | 資訊管理學研究所 | zh_TW |
顯示於系所單位: | 資訊管理學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
U0001-0508202013275700.pdf | 2.27 MB | Adobe PDF | 檢視/開啟 |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。