頻寬限制下的全息遠程呈現及3D模型重建

Abstract

隨著諸如全息遠程呈現(Holographic telepresence)的虛擬現實(VR)的應用的出現，帶來相較於傳統多媒體系統更高的資料量傳輸，然而當前網路可能不堪負荷這些應用的高頻寬要求。另一方面，相機混合場(CBF)也被提出來恢復相機的被遮擋區域以增強用戶在VR中的體驗(QoE)。然而，之前關於有效網路資源分配的研究主要集中在傳統的單視圖，多視圖和監控視頻系統上，並不能支持新穎的VR應用。在本文中，我們利用了資源選擇，以在網路容量和3D重建的約束下來最大化用戶的QoE。我們首先制定一個新的最佳化問題，稱為在具有遮擋的實時遠程呈現中做資源選擇(SSRTO)，目標是QoE最大化。接著我們證明NP-困難並提出一種新的算法，稱為最大化完整模型質量(MQCM)，通過檢查我們事先建立的角度定向圖(ADG)和鄰近定向圖(NDG)來在網路容量及即時的條件之下同時最大化用戶的QoE及重建完整的3D模型。此外，我們還設計了一個偽多項式時間最優算法，稱為最佳化完整模型質量(OQCM)，用於無遮擋發生的系統。根據模擬結果表明，在重構完整的三維模型時，無論是滿意度得分還是頻寬消耗，我們的演算法都能夠勝過過傳統的基本作法至少60%。

With the emergence of virtual reality (VR) like holographic telepresence, it is envisaged that the current networks may be overwhelmed due to the higher bandwidth demand to support high-resolution videos. On the other hand, Camera Blending Field (CBF) has been proposed to restore the occluded regions of cameras to enhance the user's Quality of Experience (QoE) in VR. However, previous works talking about effective resource allocation focused on traditional single-view, multi-view, and surveillance videos, and can not support the novel VR applications. In this paper, we explore the source selection to maximize the user's QoE under the network capacity and 3D reconstruction constraints. We first formulate a new optimization problem, named Source Selection for Real-time Telepresence with Occlusion (SSRTO), with the objective of QoE maximization. Then, we prove the NP-hardness and propose a novel algorithm, Maximum Quality of Complete Model (MQCM), to maximize the user's QoE by examining the proximity of cameras on the built Angle Directed Graph (ADG) and Neighbor Directed Graph (NDG) for reconstructing the complete 3D model. Besides, we also design a pseudo polynomial time optimal algorithm, named Optimal Quality of Complete Model (OQCM), for the occlusion-free setting. Simulation results show that our algorithms can outperform baselines no matter satisfaction score or bandwidth consumption when reconstructing complete 3D model.