Sound Blending: 探索聲音操控以實現無障礙混合現實感知

Abstract

混合實境（MR）音景將真實世界的聲音與來自聽覺設備的虛擬音效結合，呈現複雜的聽覺資訊，往往難以區分與辨識。這對視力有挑戰或視力受損的個體來說尤其具挑戰性，因為他們在日常生活中仰賴聲音和描述。為了瞭解如何處理複雜的聲音資訊，我們分析了視力受損社群內的網路討論區貼文，辨識出常見的情境、需求和所期望的解決方案。我們綜合了這些結果，並提出了「Sound Blending」的方法，旨在提高MR聲音感知，其中包含六種聲音操作：環境建構、特徵轉移、音效生成、優先處理、音源定位和風格調整。為了評估聲音混合的效果，我們進行了一項使用者實踐研究，共有18名視力受損的參與者參與，涵蓋三種模擬的MR情境，參與者需要識別複雜音景中的特定聲音。我們發現聲音混合有助提高MR聲音感知並減輕認知負擔。最後，我們開發了三個實際應用示例，以證明聲音混合的實用性。

Mixed-reality (MR) soundscapes blend real-world sound with virtual audio from hearing devices, presenting intricate auditory information that is hard to discern and differentiate. This is particularly challenging for blind or visually impaired individuals, who rely on sounds and descriptions in their everyday lives. To understand how complex audio information is consumed, we analyzed online forum posts within the blind community, identifying prevailing challenges, needs, and desired solutions. We synthesized the results and proposed Sound Blending for increasing MR sound awareness, which includes six sound manipulations: Ambience Builder, Feature Shifter, Earcon Generator, Prioritizer, Spatializer, and Stylizer. To evaluate the effectiveness of sound blending, we conducted a user enactment study with 18 blind participants across three simulated MR scenarios, where participants identified specific sounds within intricate soundscapes. We found that sound blending increased MR sound awareness and minimized cognitive load. Finally, we developed three real-world example applications to demonstrate the practicality of sound blending.