具影像遮蔽效果之單目SLAM多使用者定位系統應用於擴增實境

Abstract

近年來隨著擴增實境技術的快速發展，原先以單人體驗為主的擴增實境應用開始往多人共同體驗的場景發展，並在遊戲、教育、展覽等領域得到廣泛的應用。在多人共同體驗的相關應用開發中，如何對各使用者進行空間定位，並確保多人定位和方向的同步性及一致性是一個重大的課題。目前市面上的AR開發套件皆存在一些限制，例如Vuforia使用基於標記的定位技術，該技術可以使用標記來計算及同步每位使用者的位置及方向。然而，當使用者視線範圍中沒有標記物時，定位功能將無法繼續進行。而ARCore使用無標記定位方法，利用環境中的特徵點來計算使用者的相機姿態，但尚未有一個成熟的方法來對齊身處異地的使用者座標系，並且ARCore僅能在官方認證的AR設備上運行，對一般的擴增實境應用開發者來說造成了阻礙。

在這篇論文中，我們以Unity 3D遊戲引琴作為應用程式的開發平台，提出了一個基於ORB-SLAM2的多人定位系統，使用單目RGB影像達到使用者定位以及偵測環境中可放置虛擬物件的平面，以此虛擬物件作為多使用者定位座標同步的參考點，透過中央伺服器將定位資訊在各使用者的擴增實境設備間進行傳輸，進而以虛擬分身呈現其他使用者在空間中相對於此虛擬物件的位置及移動。此外，我們使用深度學習的技術，以單張RGB影像預估出影像的深度圖，以此解決AR應用中的遮擋問題，使得虛擬物體可以更自然的顯示在AR場景中。

在實驗方面，我們對系統中的各個模組進行定量和定性的分析，以展現這些模組的效果。此外，我們設計了一份關於本系統受試問卷，邀請數名受試者使用此AR多人定位系統，並填寫問卷以評斷系統的穩定度及體驗感受，根據受試者的問卷反饋，大多數使用者肯定此系統的穩定性及擁有不錯的體驗，並願意使用基於本系統開發的應用程式。

In recent years, with the rapid development of augmented reality (AR) technology, AR applications that were originally focused on single-user experiences have started to shift towards multi-user collaborative experiences. They have been widely applied to various fields such as gaming, education, and exhibitions. For developing multi-user experience applications, ensuring the spatial localization of every user and maintaining synchronization and consistency of positioning and orientation across multiple users is a significant challenge. Currently, available AR development kits have certain limitations. For example, Vuforia uses marker-based tracking technology, which calculates and synchronizes the position and orientation of each user based on markers. However, when there are no markers within the user's field of view, the tracking functionality cannot continue. Despite that, ARCore uses a markerless tracking method that utilizes feature points in the surrounding environment to determine the user's camera pose, yet there isn't a mature method for aligning the coordinate systems of users in different locations. Moreover, ARCore can only run on officially certified AR devices, which is a barrier for general AR application developers.

In this thesis, we propose a multi-user localization system based on ORB-SLAM2 using monocular RGB images as a development platform for application with the Unity 3D game engine. This system not only performs user localization but also places a common virtual object on a planal surface (such as table) in the environment so that every user holds a proper perspective view of the object. Note that these generated virtual objects serve as reference points for multi-user position synchronization. The positioning information is passed among every user's AR devices via a central server, based on which the relative position and movement of other users in the space of a specific user are presented via virtual avatars all with respect to these virtual objects. In addition, we use deep learning techniques to estimate the depth map of an image from a single RGB image to solve occlusion problems in AR applications, making virtual objects appear more natural in AR scenes.

In the experiment, we have conducted quantitative and qualitative analyses on each module of the system to demonstrate their effectiveness. Additionally, we have designed a questionnaire for the participants to evaluate the stability and user experience of the AR multi-user positioning system. Several participants were invited to use the system and provide feedback by filling out the questionnaire. It turns out that the experiences of most participants appear to be satisfactory, which suggest that our proposed system is highly promising.