在VR 體驗中使用基於人工智慧的具象化虛擬助理之研究

Abstract

隨著多媒體技術的進步，人們可以透過虛擬實境（VR）技術遠程探索文化遺址，甚至突破地理限制，參觀已不復存在的歷史場所。過去我們利用敦煌研究院的數據重建敦煌第61窟的內部樣貌，讓使用者透過頭戴式裝置觀看這個空間，他們可以選擇敘事性腳本的導覽或自主探索。然而，敘事性導覽可能導致體驗者參與感較低，自主探索則容易讓使用者迷失或錯過重要內容。

本文提出一種融合敘事導覽與自由探索優勢的體驗方法，透過整合虛擬代理人來為使用者提供結構化資訊，並同時保持其自主性。我們使用 Microsoft Azure 和 ChatGPT 創建了對話式虛擬代理人，並在虛擬環境中提供具象化代理人、粒子特效等視覺引導來增強使用者的整體導覽體驗。

基於敦煌第61窟虛擬導覽系統的使用者研究顯示，不同虛擬代理外觀（全身、僅頭部、靈魂狀）對使用者體驗有顯著影響。結果指出，全身代理人因其臉部表情和身體動作提供了最高的真實感和存在感，最受參與者喜愛；僅頭部的代理人因形象不完整而不受歡迎，即使其存在感高於靈魂狀代理人。

With the advancement of multimedia technology, visitors can now explore cultural heritage sites remotely with the support of various technologies. Virtual Reality (VR), in particular, stands out by offering the ability to virtually visit and reconstruct historical sites that are no longer physically present. This breakthrough allows individuals to overcome geographical constraints, enabling broader access to museum experiences. Previously, we harnessed digitized data from the Dunhuang Research Academy to recreate the internal ambiance of the Dunhuang Caves. Through head-mounted displays, users have the option of navigating these virtual spaces either via structured tours or through self-directed exploration. Each approach, however, presents challenges. Structured tours can sometimes lead to a more passive engagement, potentially reducing the user's involvement and the depth of learning. Conversely, the absence of guided tours may leave users feeling lost, potentially missing out on crucial educational content and insights.

This paper introduces an approach to virtual touring that merges the benefits of guided and exploratory experiences by integrating a virtual agent. This method aims to balance the provision of structured information with preserving user autonomy. The proposed assistive guiding system is developed using advanced technologies. Firstly, we utilize Microsoft Azure in conjunction with ChatGPT to create an AI-driven conversational virtual agent. This agent offers users clear guidance and access to extensive knowledge. Secondly, the system features embodied virtual agents and corresponding animations to provide visual cues and guidance within the virtual environment, enhancing the overall user guiding experience.

The user study was conducted based on the Dunhuang Cave VR experience using the virtual agent guiding system. The experiment aimed to understand the impact of different virtual agent appearances—full-body, head-only, and spirit-like—on user experience during Dunhuang Cave 61 VR tours. We sought to further understand the influence of facial expressions and body movements on users' interactions with the virtual agent. The results show that full-body agents with both facial expressions and body gestures provide the highest realism and presence, and are preferred by most participants. The head-only appearance is the least preferred due to its incomplete human form, although it has a higher presence than the spirit-like agent.