請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/90542
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 傅立成 | zh_TW |
dc.contributor.advisor | Li-Chen Fu | en |
dc.contributor.author | 張禾姈 | zh_TW |
dc.contributor.author | Ho-Ling Chang | en |
dc.date.accessioned | 2023-10-03T16:33:23Z | - |
dc.date.available | 2023-11-09 | - |
dc.date.copyright | 2023-10-03 | - |
dc.date.issued | 2023 | - |
dc.date.submitted | 2023-08-08 | - |
dc.identifier.citation | [1] Reisa A Sperling, Paul S Aisen, Laurel A Beckett, David A Bennett, Suzanne Craft, Anne M Fagan, Takeshi Iwatsubo, Clifford R Jack Jr, Jeffrey Kaye, Thomas J Mon- tine, et al. Toward defining the preclinical stages of alzheimer’s disease: Recommendations from the national institute on aging-alzheimer’s association workgroups on diagnostic guidelines for alzheimer’s disease. Alzheimer’s & dementia, 7(3):280– 292, 2011.
[2] Shannon L Risacher and Andrew J Saykin. Neuroimaging and other biomarkers for alzheimer’s disease: the changing landscape of early detection. Annual review ofclinical psychology, 9:621–648, 2013. [3] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances inneural information processing systems, 30, 2017. [4] Sebastian Ruder, Matthew E Peters, Swabha Swayamdipta, and Thomas Wolf.Transfer learning in natural language processing. In Proceedings of the 2019conference of the North American chapter of the association for computationallinguistics: Tutorials, pages 15–18, 2019. [5] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of theAssociation for Computational Linguistics: Human Language Technologies,Volume 1 (Long and Short Papers), pages 4171–4186, 2019. [6] Kaj Blennow, Mony J de Leon, and Henrik Zetterberg. Alzheimer’s disease. TheLancet, 368(9533):387–403, 2006. [7] Sheung-Tak Cheng. Dementia caregiver burden: a research update and critical anal- ysis. Current psychiatry reports, 19:1–8, 2017. [8] Ronald C Petersen, Oscar Lopez, Melissa J Armstrong, Thomas SD Getchius, Mary Ganguli, David Gloss, Gary S Gronseth, Daniel Marson, Tamara Pringsheim, Gre- gory S Day, et al. Practice guideline update summary: Mild cognitive impairment: Report of the guideline development, dissemination, and implementation subcom- mittee of the american academy of neurology. Neurology, 90(3):126–135, 2018. [9] Berndt Winblad, Katie Palmer, Miia Kivipelto, Vesna Jelic, Laura Fratiglioni, L-O Wahlund, Agneta Nordberg, Lars Bäckman, Michael Albert, Ove Almkvist, et al. Mild cognitive impairment–beyond controversies, towards a consensus: report of the international working group on mild cognitive impairment. Journal of internalmedicine, 256(3):240–246, 2004. [10] Marilyn S Albert, Steven T DeKosky, Dennis Dickson, Bruno Dubois, Howard H Feldman, Nick C Fox, Anthony Gamst, David M Holtzman, William J Jagust, Ronald C Petersen, et al. The diagnosis of mild cognitive impairment due to alzheimer’s disease: recommendations from the national institute on aging-alzheimer’s association workgroups on diagnostic guidelines for alzheimer’s disease. Alzheimer’s & dementia, 7(3):270–279, 2011. [11] Kenneth M Langa and Deborah A Levine. The diagnosis and management of mild cognitive impairment: a clinical review. Jama, 312(23):2551–2561, 2014. [12] Ronald C Petersen, Paul Aisen, Bradley F Boeve, Yonas E Geda, Robert J Ivnik, David S Knopman, Michelle Mielke, Vernon S Pankratz, Rosebud Roberts, Walter A Rocca, et al. Mild cognitive impairment due to alzheimer disease in the community.Annals of neurology, 74(2):199–208, 2013. [13] Sarah Tomaszewski Farias, Dan Mungas, Bruce R Reed, Danielle Harvey, and Charles DeCarli. Progression of mild cognitive impairment to dementia in clinic- vs community-based cohorts. Archives of neurology, 66(9):1151–1157, 2009. [14] Sima Ataollahi Eshkoor, Tengku Aizan Hamid, Chan Yoke Mun, and Chee Kyun Ng. Mild cognitive impairment and its management in older people. Clinicalinterventions in aging, pages 687–693, 2015. [15] Ana Luisa Sosa, Emiliano Albanese, Blossom CM Stephan, Michael Dewey, Daisy Acosta, Cleusa P Ferri, Mariella Guerra, Yueqin Huang, KS Jacob, Ivonne Z Jimenez-Velazquez, et al. Prevalence, distribution, and impact of mild cognitive im- pairment in latin america, china, and india: a 10/66 population-based study. PLoSmedicine, 9(2):e1001170, 2012. [16] Ronald C Petersen, James C Stevens, Mary Ganguli, Eric G Tangalos, Jeffrey L Cummings, and Steven T DeKosky. Practice parameter: Early detection of demen- tia: Mild cognitive impairment (an evidence-based review)[retired]: Report of the quality standards subcommittee of the american academy of neurology. Neurology, 56(9):1133–1142, 2001. [17] Marta Crous-Bou, Carolina Minguillón, Nina Gramunt, and José Luis Molinuevo.Alzheimer’s disease prevention: from risk factors to early intervention. Alzheimer’sresearch & therapy, 9(1):1–9, 2017. [18] Kaj Blennow and Henrik Zetterberg. Biomarkers for alzheimer’s disease: current status and prospects for the future. Journal of internal medicine, 284(6):643–663, 2018. [19] A Caroli, GB Frisoni, Alzheimer’s Disease Neuroimaging Initiative, et al. The dynamics of alzheimer’s disease biomarkers in the alzheimer’s disease neuroimaging initiative cohort. Neurobiology of aging, 31(8):1263–1274, 2010. [20] Anja Soldan, Corinne Pettigrew, Yuxin Zhu, Mei-Cheng Wang, Abhay Moghekar, Rebecca F Gottesman, Baljeet Singh, Oliver Martinez, Evan Fletcher, Charles De- Carli, et al. White matter hyperintensities and csf alzheimer disease biomarkers in preclinical alzheimer disease. Neurology, 94(9):e950–e960, 2020. [21] Christine Fennema-Notestine, Linda K McEvoy, Donald J Hagler Jr, Mark W Jacob- son, Anders M Dale, Alzheimer’s Disease Neuroimaging Initiative, et al. Structural neuroimaging in the detection and prognosis of pre-clinical and early ad. Behaviouralneurology, 21(1-2):3–12, 2009. [22] Linda K McEvoy, Christine Fennema-Notestine, J Cooper Roddey, Donald J Ha- gler Jr, Dominic Holland, David S Karow, Christopher J Pung, James B Brewer, and Anders M Dale. Alzheimer disease: quantitative structural neuroimaging for detection and prediction of clinical and structural changes in mild cognitive impairment.Radiology, 251(1):195–205, 2009. [23] Claudia Jacova, Andrew Kertesz, Mervin Blair, John D Fisk, and Howard H Feld- man. Neuropsychological testing and assessment for dementia. Alzheimer’s &Dementia, 3(4):299–317, 2007. [24] Saturnino Luz, Fasih Haider, Sofia de la Fuente, Davida Fromm, and Brian MacWhinney. Alzheimer’s dementia recognition through spontaneous speech: The adress challenge. Proc. Interspeech 2020, pages 2172–2176, 2020. [25] Saturnino Luz, Fasih Haider, Sofia de la Fuente, Davida Fromm, and Brian MacWhinney. Detecting cognitive decline using speech only: The adresso challenge. In INTERSPEECH 2021. ISCA, 2021. [26] Kimberly D Mueller, Bruce Hermann, Jonilda Mecollari, and Lyn S Turkstra. Connected speech and language in mild cognitive impairment and alzheimer’s dis- ease: A review of picture description tasks. Journal of clinical and experimental neuropsychology, 40(9):917–939, 2018. [27] José Vicente Egas López, László Tóth, Ildikó Hoffmann, János Kálmán, Magdolna Pákáski, and Gábor Gosztolya. Assessing alzheimer’s disease from speech using the i-vector approach. In Speech and Computer: 21st International Conference, SPECOM 2019, Istanbul, Turkey, August 20–25, 2019, Proceedings 21, pages 289–298. Springer, 2019. [28] C Huang, L-O Wahlund, T Dierks, P Julin, B Winblad, and V Jelic. Discrimination of alzheimer’s disease and mild cognitive impairment by equivalent eeg sources: a cross-sectional and longitudinal study. Clinical Neurophysiology, 111(11):1961– 1967, 2000. [29] Shannon L Risacher, Li Shen, John D West, Sungeun Kim, Brenna C McDonald, Laurel A Beckett, Danielle J Harvey, Clifford R Jack Jr, Michael W Weiner, An- drew J Saykin, et al. Longitudinal mri atrophy biomarkers: relationship to conversion in the adni cohort. Neurobiology of aging, 31(8):1401–1418, 2010. [30] Jasper D Sluimer, Femke H Bouwman, Hugo Vrenken, Marinus A Blankenstein, Frederik Barkhof, Wiesje M van der Flier, and Philip Scheltens. Whole-brain atrophy rate and csf biomarker levels in mci and ad: a longitudinal study. Neurobiology of aging, 31(5):758–764, 2010. [31] Chantel D Mayo, Erin L Mazerolle, Lesley Ritchie, John D Fisk, Jodie R Gawryluk, Alzheimer’s Disease Neuroimaging Initiative, et al. Longitudinal changes in microstructural white matter metrics in alzheimer’s disease. NeuroImage: Clinical, 13:330–338, 2017. [32] Nichole LJ Saunders and Mathew J Summers. Longitudinal deficits to attention, executive, and working memory in subtypes of mild cognitive impairment. Neuropsychology, 25(2):237, 2011. [33] Meiyan Huang, Wei Yang, Qianjin Feng, and Wufan Chen. Longitudinal measurement and hierarchical classification framework for the prediction of alzheimer’s disease. Scientific reports, 7(1):1–13, 2017. [34] Man Guo, Yongchao Li, Weihao Zheng, Keman Huang, Li Zhou, Xiping Hu, Zhijun Yao, and Bin Hu. A novel conversion prediction method of mci to ad based on longitudinal dynamic morphological features using adni structural mris. Journal of Neurology, 267:2983–2997, 2020. [35] Hao Guan, Tao Liu, Jiyang Jiang, Dacheng Tao, Jicong Zhang, Haijun Niu, Wan- lin Zhu, Yilong Wang, Jian Cheng, Nicole A Kochan, et al. Classifying mci sub- types in community-dwelling elderly using cross-sectional and longitudinal mri-based biomarkers. Frontiers in Aging Neuroscience, 9:309, 2017. [36] I Driscoll, C Davatzikos, Y An, X Wu, D Shen, M Kraut, and SM2690968 Resnick. Longitudinal pattern of regional brain volume change differentiates normal aging from mci. Neurology, 72(22):1906–1913, 2009. [37] Costas Boletsis. A review of automated speech-based interaction for cognitive screening. Multimodal Technologies and Interaction, 4(4):93, 2020. [38] Marshal F Folstein, Susan E Folstein, and Paul R McHugh. “mini-mental state": a practical method for grading the cognitive state of patients for the clinician. Journal of psychiatric research, 12(3):189–198, 1975. [39] Ziad S Nasreddine, Natalie A Phillips, Valérie Bédirian, Simon Charbonneau, Vic- tor Whitehead, Isabelle Collin, Jeffrey L Cummings, and Howard Chertkow. The montreal cognitive assessment, moca: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53(4):695–699, 2005. [40] Alexander Prange, Mira Niemann, Antje Latendorf, Anika Steinert, and Daniel Sonntag. Multimodal speech-based dialogue for the mini-mental state examination. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–8, 2019. [41] Paul Devos, Jari Debeer, Jelle Ophals, and Mirko Petrovic. Cognitive impairment screening using m-health: An android implementation of the mini-mental state ex- amination (mmse) using speech recognition. European Geriatric Medicine, 10:501– 509, 2019. [42] Simone Varrasi, Santo Di Nuovo, Daniela Conti, and Alessandro Di Nuovo. A social robot for cognitive assessment. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, pages 269–270, 2018. [43] Andrew Kertesz. Western aphasia battery test manual. Psychological Corporation, 1982. [44] James T Becker, François Boiler, Oscar L Lopez, Judith Saxton, and Karen L Mc- Gonigle. The natural history of alzheimer’s disease: description of study cohort and accuracy of diagnosis. Archives of neurology, 51(6):585–594, 1994. [45] José Vicente Egas-López, Réka Balogh, Nóra Imre, Ildikó Hoffmann, Martina Katalin Szabó, László Tóth, Magdolna Pákáski, János Kálmán, and Gábor Gosz- tolya. Automatic screening of mild cognitive impairment and alzheimer’s disease by means of posterior-thresholding hesitation representation. Computer Speech &Language, 75:101377, 2022. [46] Gábor Gosztolya, Réka Balogh, Nóra Imre, Jose Vicente Egas-Lopez, Ildikó Hoff- mann, Veronika Vincze, László Tóth, Davangere P Devanand, Magdolna Pákáski, and János Kálmán. Cross-lingual detection of mild cognitive impairment based on temporal parameters of spontaneous speech. Computer Speech & Language, 69:101215, 2021. [47] Daniela Beltrami, Laura Calzà, Gloria Gagliardi, Enrico Ghidoni, Norina Marcello, Rema Rossini Favretti, and Fabio Tamburini. Automatic identification of mild cognitive impairment through the analysis of italian spontaneous speech productions. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2086–2093, 2016. [48] Sabah Al-Hameed, Mohammed Benaissa, and Heidi Christensen. Detecting and predicting alzheimer’s disease severity in longitudinal acoustic data. In Proceedings of the International Conference on Bioinformatics Research and Applications 2017, pages 57–61, 2017. [49] Maria Yancheva, Kathleen C Fraser, and Frank Rudzicz. Using linguistic features longitudinally to predict clinical scores for alzheimer’s disease and related dementias. In Proceedings of SLPAT 2015: 6th Workshop on Speech and Language Processing for Assistive Technologies, pages 134–139, 2015. [50] Sercan Ö Arik and Tomas Pfister. Tabnet: Attentive interpretable tabular learning. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 6679–6687, 2021. [51] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014. [52] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211–252, 2015. [53] Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, page 807–814, Madison, WI, USA, 2010. [54] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprintarXiv:1606.08415, 2016. [55] Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6):82–97, 2012. [56] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. [57] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010. [58] Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747, 2016. [59] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR (Poster), 2015. [60] Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. 2018. [61] Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning. In International conference on machine learning, pages 1139–1147. PMLR, 2013. [62] Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339, 2018. [63] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. [64] Alan Akbik, Tanja Bergmann, and Roland Vollgraf. Pooled contextualized embed- dings for named entity recognition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 724–728, 2019. [65] Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In International conference on machine learning, pages 933–941. PMLR, 2017. [66] Andre Martins and Ramon Astudillo. From softmax to sparsemax: A sparse model of attention and multi-label classification. In International conference on machine learning, pages 1614–1623. PMLR, 2016. [67] Thomas Leyhe, Stephan Müller, Monika Milian, Gerhard W Eschweiler, and Ralf Saur. Impairment of episodic and semantic autobiographical memory in patients with mild cognitive impairment and early alzheimer’s disease. Neuropsychologia, 47(12):2464–2469, 2009. [68] Alexandra Barnabe, Victor Whitehead, Randi Pilon, Geneviève Arsenault-Lapierre, and Howard Chertkow. Autobiographical memory in mild cognitive impairment and alzheimer’s disease: a comparison between the levine and kopelman interview methodologies. Hippocampus, 22(9):1809–1825, 2012. [69] Mohamad El Haj, Pascal Antoine, Jean Louis Nandrino, and Dimitrios Kapogiannis. Autobiographical memory decline in alzheimer’s disease, a theoretical and clinical overview. Ageing research reviews, 23:183–192, 2015. [70] Matthew D Grilli, Aubrey A Wank, John J Bercel, and Lee Ryan. Evidence for reduced autobiographical memory episodic specificity in cognitively normal middle-aged and older individuals at increased risk for alzheimer’s disease dementia. Journal of the International Neuropsychological Society, 24(10):1073–1083, 2018. [71] Marcia K Johnson, Mary A Foley, Aurora G Suengas, and Carol L Raye. Phenomenal characteristics of memories for perceived and imagined autobiographical events. Journal of Experimental Psychology: General, 117(4):371, 1988. [72] Florian Eyben, Martin Wöllmer, and Björn Schuller. Opensmile: the munich versa- tile and fast open-source audio feature extractor. In Proceedings of the 18th ACM international conference on Multimedia, pages 1459–1462, 2010. [73] Florian Eyben, Klaus R Scherer, Björn W Schuller, Johan Sundberg, Elisabeth André, Carlos Busso, Laurence Y Devillers, Julien Epps, Petri Laukka, Shrikanth S Narayanan, et al. The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing. IEEE transactions on affective computing, 7(2):190–202, 2015. [74] Sheng-Ya Lin. Contrast-enhanced automatic cognitive impairment detection system embedded with pause encoding. Master’s thesis, National Taiwan University, 2022. [75] Hongyu Guo, Yongyi Mao, and Richong Zhang. Augmenting data with mixup for sentence classification: An empirical study. arXiv preprint arXiv:1905.08941, 2019. [76] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmentation for consistency training. Advances in Neural Information Processing Systems, 33:6256–6268, 2020. [77] Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, and Songlin Hu. Conditional bert contextual augmentation. In International Conference on Computational Science, pages 84–95. Springer, 2019. [78] Dominika Woszczyk, Anna Hedlikova, Alican Akman, Soteris Demetriou, and Björn Schuller. Data Augmentation for Dementia Detection in Spoken Language. In Proc. Interspeech 2022, pages 2858–2862, 2022. [79] Qingyu Zhao, Zixuan Liu, Ehsan Adeli, and Kilian M Pohl. Longitudinal self-supervised learning. Medical image analysis, 71:102051, 2021. [80] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33:18661–18673, 2020. | - |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/90542 | - |
dc.description.abstract | 隨著全球老年人口的增加,醫療保健系統面臨著應對日益增多的阿茲海默病患者的負擔。鑑於對治療和早期診斷的巨大需求,對於認知障礙篩查系統的廣泛研究已經展開,旨在協助醫療專業人員準確診斷阿茲海默病。本論文提出了一種多模態早期認知障礙檢測系統,利用自動提取的預定義聲學特徵和自設的嵌入來增強語言表示。該系統利用自發性非結構化語音數據,來自自傳性記憶(AM)測試,該測試是評估個體認知狀態的神經心理學評估工具。具體而言,我們的關注點是檢測輕度認知障礙(MCI),它代表健康個體和患有阿茲海默病(AD)的個體之間的中間階段。通過解決輕度認知障礙檢測問題,我們的目標是促進早期治療干預。鑑於輕度認知障礙患者表現出的症狀較輕微,整合多模態數據可以有效豐富特徵並幫助模型學習。考慮到自發性語音的非結構性和隱含性特點,我們引入了兩個額外的嵌入層,即說話者嵌入和對話嵌入,以增強模型學習的信息。為了評估我們提出的方法的有效性,我們在一個中文數據集上進行了實驗,平均準確率達到了78%。此外,我們進行了一系列消融實驗,以評估我們系統中每個模塊的貢獻。此外,我們擴展了我們的研究範圍,對使用自傳性記憶測試的非結構化語音數據進行縱向分析,這是一個尚未得到廣泛探索的研究領域。為了便於縱向分析,我們設計了一個系統,其中包括一個方向編碼器,用於學習不同訪問之間的時間信息。這種方法在至少具有兩次訪問的子數據集上顯示了3%的準確率改善。 | zh_TW |
dc.description.abstract | With the increasing global elderly population, healthcare systems face the growing burden of addressing the rising number of individuals affected by Alzheimer's disease. Given the significant demand for treatment and early diagnosis, extensive research has been conducted on cognitive impairment screening systems to assist healthcare professionals in accurately diagnosing Alzheimer's disease. This thesis proposes a multi-modal early cognitive impairment detection system that leverages automatically extracted pre-defined acoustic features and self-designed embeddings to enhance linguistic representation. The proposed system uses spontaneous unstructured speech data from autobiographical memory (AM) tests, which serve as neuropsychological assessments for evaluating individuals' cognitive states. In particular, our focus lies in detecting mild cognitive impairment (MCI), which represents the intermediate stage between healthy individuals and those with Alzheimer's disease (AD). By addressing MCI detection, we aim to facilitate early treatment interventions. Given the subtle symptoms exhibited by individuals with MCI, integrating multi-modal data can effectively enrich features and aid in model learning. Considering the unstructured and implicit nature of spontaneous speech, we introduce two additional embeddings, namely speaker embedding and conversation embedding, to augment the information available for model learning. In order to assess the efficacy of our proposed approach, we conducted experiments on a Chinese dataset, attaining an average accuracy of 78%, which is comparable to the results obtained. Moreover, we conducted a set of ablation studies to evaluate the individual contributions of each module integrated into our system. Moreover, we extend our investigation to encompass the longitudinal analysis of MCI detection using unstructured speech data from AM tests, representing a research area that has yet to be extensively explored. To facilitate longitudinal analysis, we design a system incorporating a direction encoder for learning temporal information between different visits. This approach shows an accuracy improvement of 3% on a subset of the dataset comprising subjects with at least two visits. | en |
dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2023-10-03T16:33:23Z No. of bitstreams: 0 | en |
dc.description.provenance | Made available in DSpace on 2023-10-03T16:33:23Z (GMT). No. of bitstreams: 0 | en |
dc.description.tableofcontents | 誌謝 i
中文摘要 iii ABSTRACT v CONTENTS vii LIST OF FIGURES xi LIST OF TABLES xiii Chapter 1 Introduction 1 1.1 Background 1 1.2 Motivation 4 1.3 Challenges 5 1.3.1 Sensitivity in Early Cognitive Decline 5 1.3.2 Unstructured Speech 5 1.3.3 Longitudinal Analysis 6 1.4 Related Work 7 1.4.1 Cognitive Detection based on Speech 7 1.4.1.1 Structured Speech 7 1.4.1.2 Semi-structured Speech 8 1.4.1.3 Unstructured Speech 8 1.4.2 Longitudinal Study 9 1.5 Objectives 10 1.6 Thesis Organization 11 Chapter 2 Preliminaries 13 2.1 Deep Neural Network 13 2.1.1 Neural Network 14 2.1.2 Activation Function 15 2.1.3 Loss Functions 17 2.1.4 Optimizer 19 2.2 Pre-training and Fine-tuning framework with Language Models 20 2.2.1 Self-attention Mechanism 21 2.2.2 Pre-trained Language Models for Downstream Tasks 22 2.2.3 Bidirectional Encoder Representations from Transformers 22 2.3 TabNet 24 2.3.1 Gated Linear Unit 24 2.3.2 Attentive Transformer 25 2.3.3 Feature Transformer 25 2.3.4 The Overall Model Architecture 26 Chapter 3 Methodology 28 3.1 System Overview 28 3.2 Data Collection 29 3.2.1 Recall 30 3.2.2 Probing 30 3.3 Problem Setting and Formulation 31 3.3.1 Cross-sectional Scenario 31 3.3.2 Longitudinal Scenario 32 3.4 Acoustic Feature Extraction 33 3.5 Transcript Reformatting 34 3.6 Data Augmentation 35 3.7 Multi-modal Fusion Model 36 3.7.1 Acoustic Encoder 36 3.7.2 Linguistic Encoder 37 3.7.2.1 Speaker-aware Embedding Layer 38 3.7.2.2 Conversation-aware Embedding Layer 39 3.7.3 Fusion Layer and Classifier 40 3.8 Longitudinal Analysis 40 Chapter 4 Experiments 43 4.1 Experiment Setup 43 4.1.1 Datasets 43 4.1.1.1 NTU-AM-MM Dataset 44 4.1.1.2 NTU-AM-LG Dataset 45 4.1.2 Evaluation Settings 45 4.1.3 Evaluation Metrics 46 4.1.4 Baselines 47 4.2 Implementation Details 48 4.3 NTU-AM-MM Dataset 48 4.3.1 Evaluation of Linguistic Model 49 4.3.2 Evaluation of Acoustic Model 50 4.3.3 Evaluation of Multi-modal Model 51 4.3.4 Ablation Analysis 52 4.3.5 Visualization 53 4.4 NTU-AM-LG Dataset 54 4.4.1 Results 54 4.4.2 Ablation Analysis 56 4.5 Hyper-parameters 59 Chapter 5 Conclusion 61 REFERENCE 64 | - |
dc.language.iso | en | - |
dc.title | 基於深度學習之多模態自發語言 早期認知障礙檢測系統 | zh_TW |
dc.title | Multi-modal Early Cognitive Impairment Detection System for Spontaneous Speech using Deep Learning | en |
dc.type | Thesis | - |
dc.date.schoolyear | 111-2 | - |
dc.description.degree | 碩士 | - |
dc.contributor.oralexamcommittee | 張玉玲;邱銘章;李宏毅;林澤 | zh_TW |
dc.contributor.oralexamcommittee | Yu-Ling Chang;Ming-Jang Chiu;Hung-Yi Lee;Che Lin | en |
dc.subject.keyword | 多模態學習,輕度認知功能障礙,自發語言,縱向分析,快篩系統, | zh_TW |
dc.subject.keyword | Multi-modal learning,Mild cognitive impairment,Spontaneous speech,Longitudinal analysis,Screening system, | en |
dc.relation.page | 76 | - |
dc.identifier.doi | 10.6342/NTU202302644 | - |
dc.rights.note | 同意授權(限校園內公開) | - |
dc.date.accepted | 2023-08-10 | - |
dc.contributor.author-college | 電機資訊學院 | - |
dc.contributor.author-dept | 資訊網路與多媒體研究所 | - |
顯示於系所單位: | 資訊網路與多媒體研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-111-2.pdf 目前未授權公開取用 | 8.97 MB | Adobe PDF | 檢視/開啟 |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。