無聲話語介面裝置之人工智慧模型的開發

Abstract

無聲語音介面（SSI）能在無法獲取語音訊號的情況下辨識口語發音，並可作為語音障礙患者的溝通翻譯工具。本研究針對語言障礙患者的溝通需求，開發了一套人工智慧無聲話語介面系統。該系統結合可穿戴式感測裝置與人工智慧無聲話語辨識模型，能夠擷取人體面部的肌肉電訊號（EMG），並透過導電高分子應變感測器量測穿戴式支架上的彎曲應變。在系統訊號轉換上，所獲取的肌電和應變訊號首先會先進行前處理，隨後利用卷積神經網路（CNN）進行特徵萃取，捕捉原始訊號中的時域與頻域特徵。接著，這些特徵被輸入至Transformer神經網路，用以建立訊號序列與語意之間的對應關係，將非聲學的訊號轉換為對應的詞彙特徵。最後，系統整合連接主義時序分類(CTC)演算法及n-gram語言模型，進行更精確的詞彙特徵解碼與語句預測，實現中英文語音的文字輸出。系統於中英文多組常用詞彙上進行驗證，分別於英文796筆與中文592筆詞彙下達到8.6%與13.11%的文字錯誤率（WER），多組常用詞彙辨識準確率亦均超過95%。整體而言，本研究所提出的AI無聲話語介面系統展現出優異的跨語言辨識效能，不僅為語言障礙者或特殊環境下的無聲話語溝通提供創新解決方案，更具備推廣至智慧醫療及人機互動等領域的高度應用潛力。

Silent speech interfaces (SSIs) enable recognition of verbal expressions from non-acoustic biosignals generated by facial movements, offering a promising solution for people with voice disorders or difficulties in verbal communication. In this work, we present a wearable silent speech interface (SSI) system integrating electromyogram (EMG) sensing electrodes with a conductive polymer-based strain sensor. An AI speech recognition model processes these signals to enable assisted speaking without relying on vocal fold activation. After preprocessing the acquired EMG and strain signals for obtaining high-quality input for recognition, features are extracted using a convolutional neural network (CNN), and a transformer architecture is utilized to aggregate contextual features into meaningful word features. The feature outputs are passed through a connectionist temporal classification (CTC) decoder, which aligns the predicted sequences with target sentence labels and, combined with an n-gram language model, accurately generates the final word sequences for both Chinese and English words. The AI speech recognition model achieved excellent word error rates (WER) of 8.6% and 13.11% on datasets containing 796 English and 592 Chinese words, demonstrating high recognition accuracy (>95%) across various datasets of commonly used vocabulary. This proposed wearable silent speech interface (SSI) potentially helps people with vocal cord injuries regain their ability to speak and enables effective communication in special situations and environments.