文字氣泡框：聽障者之多人對話情境下的視覺化設計

Abstract

如何在 VR/AR 的三維空間中顯示即時的對話文字一直是個重要的 研究議題。我們的目標是提出在 VR/AR 空間中的對話視覺化方法，將 語音資訊最佳化的放置在合適的位置使之變成動態的即時字幕系統。 在研究過程中，我們發現聽障者在多人對話情境下，對於這樣的視覺 化方法有更急迫的需求，他們在多人對話情境與聽力正常者對話時時 常遇到諸如:無法將字幕對應上說話者、無法應對來自視野外的話語 等等困難。我們對應了這些困難提出了多種設計並且透過十二位聽障 使用者測試進行評估。評估的結果顯示比起傳統的字幕顯示，使用者 比較喜歡能夠明確聯繫話語與說話者的氣泡框設計。我們根據這次使 用者測試的結果開發了「文字氣泡框」，一個 AR 語音即時辨識介面。 經過我們的使用評測，在多人交談情境下，比起傳統字幕顯示系統， 大多數聽障使用者比較傾向使用我們的系統。

Deaf and hard-of-hearing (DHH) individuals encounter difficulties when engaged in group conversations with hearing individuals, due to factors such as simultaneous utterances from multiple speakers and speakers whom may be potentially out of view. We interviewed and co-designed with eight DHH participants to address the following challenges: 1) associating utterances with speakers, 2) ordering utterances from different speakers, 3) displaying optimal content length, and 4) visualizing utterances from out-of-view speakers. We evaluated multiple designs for each of the four challenges through a user study with twelve DHH participants. Our study results showed that participants significantly preferred speech bubble visualizations over traditional captions. These design preferences guided our development of SpeechBubbles, a real-time speech recognition interface prototype on an augmented reality head-mounted display. From our evaluations, we further demonstrated that DHH participants preferred our prototype over traditional captions for group conversations.