請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/93305完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 邱振豪 | zh_TW |
| dc.contributor.advisor | Chenhao Chiu | en |
| dc.contributor.author | 黃建誌 | zh_TW |
| dc.contributor.author | Jian-Zhi Huang | en |
| dc.date.accessioned | 2024-07-29T16:09:35Z | - |
| dc.date.available | 2024-07-30 | - |
| dc.date.copyright | 2024-07-29 | - |
| dc.date.issued | 2024 | - |
| dc.date.submitted | 2024-07-23 | - |
| dc.identifier.citation | Ashokumar, M., Guichet, C., Schwartz, J. L., & Ito, T. (2023). Correlation between the effect of orofacial somatosensory inputs in speech perception and speech production performance. Auditory Perception & Cognition, 6(1-2), 97-107.
Bicevskis, K., Derrick, D., & Gick, B. (2016). Visual-tactile integration in speech perception: Evidence for modality neutral speech primitives. The Journal of the Acoustical Society of America, 140(5), 3531-3539. Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01 Boersma, Paul & Weenink, David (2023). Praat: doing phonetics by computer [Computer program]. Retrieved from http://www.praat.org/ Burr, D., & Gori, M. (2012). “Multisensory integration develops late in humans,” in The Neural Bases of Multisensory Processes. eds. M. M. Murray and M. T. Wallace (Boca Raton, FL: CRC Press), 345–362. Chao, S. C., Ochoa, D., & Daliri, A. (2019). Production variability and categorical perception of vowels are strongly linked. Frontiers in Human Neuroscience, 13, 96. Chiu, C., Huang, J.-Z., & Huang, P.-H. (2023). Perceptual identification of high vowels in Taiwan Mandarin. The Journal of the Acoustical Society of America, 154, A35. Chiu, C., & Huang, P.-H. (2023). Lip postures of high vowels in Taiwan Mandarin. Proceedings of the 2023 International Congress of Phonetic Sciences, 1052 – 1056. Chiu, C., Chang, C.-H., Huang, J.-Z., & Huang, P.-H. (2024). Enhancing lip contrasts between /u/ and /y/ in Taiwan Mandarin. The 13th International Seminar on Speech Production, May. 14–17, Autrans, France. Derrick, D., Hansmann, D., & Theys, C. (2019). Tri-modal speech: Audio-visual-tactile integration in speech perception. The Journal of the Acoustical Society of America, 146(5), 3495-3504. Dietziker, J., Staib, M., & Frühholz, S. (2021). Neural competition between concurrent speech production and other speech perception. Neuroimage, 228, 117710. Fridriksson, J., Baker, J. M., Whiteside, J., Eoute Jr, D., Moser, D., Vesselinov, R., & Rorden, C. (2009). Treating visual speech perception to improve speech production in nonfluent aphasia. Stroke, 40(3), 853-858. Fridriksson, J., Hubbard, H. I., Hudspeth, S. G., Holland, A. L., Bonilha, L., Fromm, D., & Rorden, C. (2012). Speech entrainment enables patients with Broca’s aphasia to produce fluent speech. Brain, 135(12), 3815-3829. Gick, B., & Derrick, D. (2009). Aero-tactile integration in speech perception. Nature, 462(7272), 502-504. Gick, B., Wilson, I., & Derrick, D. (2013). Articulatory phonetics. John Wiley & Sons. Honda, K., Kurita, T., Kakita, Y., & Maeda, S. (1995). Physiology of the lips and modelingof lip gestures. Journal of Phonetics, 23(1-2), 243-254. Huang, J.-H., & Chiu, C. (2023). The role of feedback in lip-tube perturbation of Taiwan Mandarin rounded vowels. Proceedings of the 2023 International Congress of Phonetic Sciences, 1037 – 1041. Huettig, F., & Hartsuiker, R. J. (2010). Listening to yourself is like listening to others: External, but not internal, verbal self-monitoring is based on speech perception. Language and Cognitive Processes, 25(3), 347-374. Ito, T., Tiede, M., & Ostry, D. J. (2009). Somatosensory function in speech perception. Proceedings of the National Academy of Sciences, 106(4), 1245-1248. Jack, B. N., Le Pelley, M. E., Han, N., Harris, A. W., Spencer, K. M., & Whitford, T. J. (2019). Inner speech is accompanied by a temporally-precise and content-specific corollary discharge. Neuroimage, 198, 170-180. Kawahara, H., Morise, M., Takahashi, T., Nisimura, R., Irino, T., & Banno, H. (2008, March). Tandem-STRAIGHT: A temporally stable power spectral representation for periodic signals and applications to interference-free spectrum, F0, and aperiodicity estimation. In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 3933-3936). IEEE. Keough, M., Derrick, D., & Gick, B. (2019). Cross-modal effects in speech perception. Annual review of linguistics, 5, 49-66. Kuhl, P. K., & Meltzoff, A. N. (1982). The bimodal perception of speech in infancy. Science, 218(4577), 1138-1141. Lametti, D. R., Nasir, S. M., & Ostry, D. J. (2012). Sensory preference in speech production revealed by simultaneous alteration of auditory and somatosensory feedback. Journal of Neuroscience, 32(27), 9351-9358. Lindblom, B. E., & Sundberg, J. E. (1971). Acoustical consequences of lip, tongue, jaw, and larynx movement. The Journal of the Acoustical Society of America, 50(4B), 1166-1179. Lisker, L., & Rossi, M. (1992). Auditory and visual cueing of the [±rounded] feature of vowels. Language and speech, 35(4), 391-417. Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., ... & Grundmann, M. (2019). Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172. Ménard, L., Trudeau-Fisette, P., & Tiede, M. K. (2022). Intelligibility of speech produced by sighted and blind adults. Plos one, 17(9), e0272127. Ménard, L., Beaudry, L., & Perrier, P. (2023). Effects of somatosensory perturbation on the perception of French /u/. JASA Express Letters, 3(5). Masapollo, M., Polka, L., & Ménard, L. (2017). A universal bias in adult vowel perception–By ear or by eye. Cognition, 166, 358-370. Masapollo, M., & Guenther, F. H. (2019). Engaging the articulators enhances perception of concordant visible speech movements. Journal of Speech, Language, and Hearing Research, 62(10), 3679-3688. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264(5588), 746-748. Ouni, S., Cohen, M. M., Ishak, H., & Massaro, D. W. (2006). Visual contribution to speech perception: measuring the intelligibility of animated talking heads. EURASIP Journal on Audio, Speech, and Music Processing, 2007, 1-12. Peirce, J. W., Gray, J. R., Simpson, S., MacAskill, M. R., Höchenberger, R., Sogo, H., Kastman, E., Lindeløv, J. (2019). PsychoPy2: experiments in behavior made easy. Behavior Research Methods. 10.3758/s13428-018-01193-y R Core Team (2017) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/ Robert-Ribes, J., Schwartz, J. L., Lallouache, T., & Escudier, P. (1998). Complementarity and synergy in bimodal speech: Auditory, visual, and audio-visual identification of French oral vowels in noise. The Journal of the Acoustical Society of America, 103(6), 3677-3689. Rosenblum, L. D. (2005). Primacy of multimodal speech perception. The handbook of speech perception, 51-78. Rosenblum, L. D. (2008). Speech perception as a multimodal phenomenon. Current directions in psychological science, 17(6), 405-409. Sams, M., Möttönen, R., & Sihvonen, T. (2005). Seeing and hearing others and oneself talk. Cognitive Brain Research, 23(2-3), 429-435. Sato, M., Troille, E., Ménard, L., Cathiard, M. A., & Gracco, V. (2013). Silent articulation modulates auditory and audiovisual speech perception. Experimental brain research, 227, 275-288. Schwartz, J. L., Berthommier, F., & Savariaux, C. (2004). Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition, 93(2), B69-B78. Scott, M. (2012). Speech imagery as corollary discharge (Doctoral dissertation, University of British Columbia). Scott, M., Yeung, H. H., Gick, B., & Werker, J. F. (2013). Inner speech captures the perception of external speech. The Journal of the Acoustical Society of America, 133(4), EL286-EL292. Siegel, G. M., & Pick, H. L. (1974). Auditory feedback in the regulation of voice. The Journal of the Acoustical Society of America, 56(5), 1618-1624. Traunmüller, H., & Öhrström, N. (2007). Audiovisual perception of openness and lip rounding in front vowels. Journal of Phonetics, 35(2), 244-258. Trudeau-Fisette, P., Ito, T., & Ménard, L. (2019). Auditory and somatosensory interaction in speech perception in children and adults. Frontiers in Human Neuroscience, 13, 344. Trudeau-Fisette, P., Arnaud, L., & Ménard, L. (2022). Visual Influence on Auditory Perception of Vowels by French-Speaking Children and Adults. Frontiers in Psychology, 13, 740271. Valkenier, B., Duyne, J. Y., Andringa, T. C., & Baskent, D. (2012). Audiovisual perception of congruent and incongruent Dutch front vowels. Van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences, 102(4), 1181-1186. Venezia, J. H., Fillmore, P., Matchin, W., Isenberg, A. L., Hickok, G., & Fridriksson, J. (2016). Perception drives production across sensory modalities: A network for sensorimotor integration of visual speech. NeuroImage, 126, 196-207. Wieling, M. (2018). Analyzing dynamic phonetic data using generalized additive mixed modeling: A tutorial focusing on articulatory differences between L1 and L2 speakers of English. Journal of Phonetics, 70, 86-116. Wood, S. (1986). The acoustical significance of tongue, lip, and larynx maneuvers in rounded palatal vowels. The Journal of the Acoustical Society of America, 80(2), 391-401. Yeung, H. H., & Werker, J. F. (2013). Lip movements affect infants’ audiovisual speech perception. Psychological Science, 24(5), 603-612. Yeung, H. H., & Scott, M. (2021). Postural control of the vocal tract affects auditory speech perception. Journal of Experimental Psychology: General, 150(5), 983. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/93305 | - |
| dc.description.abstract | 語音是多模態的，不僅概括視覺還有體感的資訊來幫助我們對於語音理解。唇部作為多模態的發音器官，提供了一個很好的機會來探索這些感官系統之間的複雜交互作用。本論文旨在研究多模態唇部資訊是否影響台灣華語中的高圓唇元音的感知辨識。此外，還探討在高圓唇元音情境下的跨模態資訊整合及交互作用。每項實驗招募了三十位台灣華語母語者，分別進行三個實驗以檢驗：1）體感回饋對聽覺感知的影響，2）視覺資訊對聽覺感知的影響，以及3）體感和視覺資訊的一致性對聽覺感知的影響。受試者透過進行內在語言表現作為體感回饋（實驗一）、觀看視覺元音影片（實驗二），或在觀看視覺元音影片的同時進行內在語言表現（實驗三），以辨識從 /u/ 到 /y / 連續體的聽覺刺激。結果顯示，儘管 /u/ 和 /y/ 在舌位上有本質的不同，體感回饋和來自唇部的視覺資訊都能顯著影響高圓唇元音 /u/ 和 /y/ 的辨別。當兩種模態提供一致的信息時，跨模態資訊的交互作用顯示出正向的累加效果。此外，當兩種模態的信息不一致時，體感對比較大的人可能會更依賴體感回饋，而體感對比較小的人可能會更依賴視覺信息。總體來說，這些發現有助於我們理解語音感知以及揭示多模態整合本身的複雜性。 | zh_TW |
| dc.description.abstract | Speech is multimodal, incorporating not only vision but also the somatosensory system to enhance understanding. The lips, being a multimodal articulator, offer a unique window to explore the complex interactions across these sensory systems. This thesis focuses on investigating whether multimodal lip information influences the identification of high rounded vowels in Taiwan Mandarin. Additionally, it aims to explore the interactions of cross-modal information under the context of high rounded vowels. Thirty native Taiwan Mandarin speakers were recruited for each of the three experiments to examine: 1) the effect of somatosensory feedback on auditory perception, 2) the effect of visual information on auditory perception, and 3) the congruent effect of somatosensory and visual information on auditory perception. They were instructed to perform inner speech as somatosensory feedback (Experiment 1), watch visually articulated vowel videos (Experiment 2), or perform inner speech with the presence of visually articulated vowel videos (Experiment 3) for the respective three experiments while identifying the auditory stimuli from /u/ - /y/ continua. The results showed that both somatosensory feedback and visual information from the lips can significantly influence the categorization of the high rounded vowels /u/ and /y/, despite their intrinsic differences in tongue position. The interaction of cross-modal information showed an additive effect when congruent information from the two modalities was provided. Additionally, individuals with greater somatosensory contrasts may rely more on somatosensory feedback when incongruent information from the two modalities was provided, whereas individuals with lesser somatosensory contrasts may rely more on visual information in the same conditions. Overall, these findings contribute to our understanding of how we perceive speech, shedding light on the complexity of multimodal integration. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2024-07-29T16:09:35Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2024-07-29T16:09:35Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | 誌謝 i
中文摘要 ii ABSTRACT iii CONTENTS iv LIST OF FIGURES vii LIST OF TABLES ix Chapter 1 Introduction 1 1.1 Research background 2 1.2 Aims of the study 2 1.3 Organization 4 Chapter 2 Literature review 6 2.1 Multimodality in speech perception 6 2.1.1 Multimodal integration and visual influence on auditory perception 6 2.1.2 Somatosensory influence on auditory perception and its interaction with vision and audition 7 2.2 Lips and multimodal feedback 8 2.2.1 Multimodal perception of roundedness 11 2.2.2 Lip postures in Taiwan Mandarin high rounded vowels 12 2.3 Research gap 15 Chapter 3 Experiment 1 21 3.1 Methods 22 3.1.1 Participants 22 3.1.2 Stimuli 23 3.1.3 Tasks 24 3.1.4 Procedure 26 3.1.5 Data analyses 28 3.2 Results 29 3.2.1 The effect of mouthing 29 3.2.2 The effect of imagining 34 3.3 Discussion 38 3.3.1 Somatosensory integration with audition 38 3.3.2 Engagement of muscles 40 3.3.3 High rounded vowels as targets of inner speech 41 3.3.4 Dimensional difference of high rounded vowels and the independence of lip postures 44 Chapter 4 Experiment 2 46 4.1 Methods 46 4.1.1 Participants 46 4.1.2 Stimuli 47 4.1.3 Tasks 51 4.1.4 Procedure 51 4.1.5 Data analyses 53 4.2 Results 54 4.3 Discussion 58 4.3.1 The effect of audio-visual integration 58 4.3.2 Visual saliency of lip postures 60 Chapter 5 Experiment 3 62 5.1 Methods 62 5.1.1 Participants 62 5.1.2 Stimuli 63 5.1.3 Tasks 63 5.1.4 Procedure 64 5.1.5 Data analyses 67 5.2 Results 70 5.2.1 The integrated effect of mouthing and visual information 70 5.2.2 The integrated effect of imagining and visual information 74 5.2.3 Correlation between the postural differences and incongruent conditions 78 5.3 Discussion 80 5.3.1 Congruent integration of somatosensory and visual information 80 5.3.2 Incongruent integration of somatosensory and visual information 81 5.3.3 Modality preference and perception-production link 83 Chapter 6 General discussion 86 6.1 Summary of findings 86 6.1.1 Bi-modal integration 86 6.1.2 Tri-modal integration 88 6.1.3 Lip postures as a multimodal gesture-specific speech content 91 6.2 Limitations and future directions 93 6.3 Conclusion 95 REFERENCE 96 Appendix A—Grouping of Experiment 3 102 | - |
| dc.language.iso | en | - |
| dc.subject | 多模態整合 | zh_TW |
| dc.subject | 語音感知 | zh_TW |
| dc.subject | 唇形姿態 | zh_TW |
| dc.subject | 高圓唇元音 | zh_TW |
| dc.subject | 台灣華語 | zh_TW |
| dc.subject | lip postures | en |
| dc.subject | multimodal integration | en |
| dc.subject | speech perception | en |
| dc.subject | Taiwan Mandarin | en |
| dc.subject | high rounded vowels | en |
| dc.title | 台灣華語圓唇元音感知中的體感與視覺影響 | zh_TW |
| dc.title | Somatosensory and Visual Influences on the Perception of Taiwan Mandarin Rounded Vowels | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 112-2 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 甯俐馨;張詠翔 | zh_TW |
| dc.contributor.oralexamcommittee | Li-Hsin Ning;Yung-hsiang Shawn Chang | en |
| dc.subject.keyword | 多模態整合,語音感知,唇形姿態,高圓唇元音,台灣華語, | zh_TW |
| dc.subject.keyword | multimodal integration,speech perception,lip postures,high rounded vowels,Taiwan Mandarin, | en |
| dc.relation.page | 102 | - |
| dc.identifier.doi | 10.6342/NTU202402131 | - |
| dc.rights.note | 同意授權(全球公開) | - |
| dc.date.accepted | 2024-07-25 | - |
| dc.contributor.author-college | 文學院 | - |
| dc.contributor.author-dept | 語言學研究所 | - |
| 顯示於系所單位： | 語言學研究所 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-112-2.pdf | 8.21 MB | Adobe PDF | 檢視/開啟 |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。