AR遠端會議中的即時摘要與重播助理

Abstract

自疫情以後，遠端會議成為部分工作的常態，現在主流的會議系統如 Google Meet 與 Microsoft Teams 被廣泛使用。而隨著 AR/VR 技術的進步，虛擬會議能夠得到進一步的提升，並提供高度沉浸感與互動性的體驗給使用者。 傳統會議中，通常會有人整理會議並紀錄以方便事後釐清內容；而現在視訊會議也有逐字稿等工具將講者的話轉錄下來，但在現在 VR/AR 的會議內，缺少紀錄並輔助使用者的工具。因此我們提出了 ARM-RSPA，一個即時紀錄會議內容與虛擬環境狀態，並根據使用者要求摘要且重播關鍵的互動的助手系統。透過 AR 重播，ARM-RSPA 提供使用者更加方便且易於理解的方法。 在後面的實驗中，ARM-RSPA 展現出在會議中的運用，並成功幫助使用者理解講者所講解的內容。我們也探討 ARM-RSPA 與其他遠端會議差別以及未來 AR 會議工具可能的發展。

Since the pandemic, remote meetings have become the norm for some work environments, with mainstream meeting systems like Google Meet and Microsoft Teams being widely used. With advancements in AR/VR technology, virtual meetings can be further enhanced to provide users with highly immersive and interactive experiences. In traditional meetings, someone typically organizes and records the meeting to clarify content afterward. While video conferences now have tools like transcripts to record speakers' words, VR/AR meetings lack tools to help with the recording and assist users. To address this challenge, we develop ARM-RSPA in the research work, which is a real-time assistant system that records meeting content and virtual environment states, summarizes based on user requests, and replays key interactions. Through AR replay, ARM-RSPA provides users with a more convenient and easily understandable way to realize the content of the meetings. To validate our research, we have conducted various experiments, where ARM-RSPA demonstrates its application in meetings and successfully helps users understand the content explained by speakers. We also discuss the differences between ARM-RSPA and other remote meeting solutions to highlight the high promise of our proposal system. Finally, we also mention our potential developments for other AR meeting tools in the future.