即時生成360 度虛擬實境全景影片之中央窩影像串接法

Abstract

近年來，虛擬實境(VR) 成為時下最迷人的技術, 尤其是沈浸式虛擬實境更是成為眾所矚目的焦點。而要生成這樣的沈浸式虛擬實境的內容，通常必須在現實的場景中利用360 度全景拍攝的方式來產生。儘管現在已經有許多拍攝裝置可以使用，但若是在高畫質的狀態下，由於運算量非常龐大，要即時地拍攝360 度全景影像並以高畫質顯示仍是非常有挑戰性的。在此我們提出了名為中央窩影像串接法的框架，定義了如何決定影像中的各個部份需要以多高的畫質去處理的方法。在這框架中主要可以分為兩個部份，其一是以人眼視覺的理論基礎去定義的敏銳程度映射函數，其二是基於影像內容對人類視覺的顯著程度來定義的顯著程度映射函數。我們的方法可以以多臺相機拍攝的內容作為輸入，即時地串接成高畫質的全景影片並串流到客戶端的裝置上。速度方面，我們使用了圖形處理器來平行化演算法已達到即時運算的層級。畫質方面，我們做了使用者經驗調查來證明我們產生出的全景影像的畫質並不因為加速而有顯著的下降。我們最終實做了我們的系統於Google Cardboard 上，並在速度上相較於原方法有六倍以上的提昇。

In recent years, virtual reality (VR) becomes one of the most fascinating technologies where real-time immersion experience is in the spotlight. In those applications, the contents are usually generated by creating a 360◦video panorama of a real-world scene. Despite that many capture devices are released, getting high-resolution panoramas and display of a virtual world at real-time update rates are still very challenging since it is a computationally demanding paradigm. In this paper, we proposed a real-time 360◦video foveated stitching framework, indicating what objects in a scene should be represented at what detail level. Our foveated stitching consists of two major parts; the acuity map and the saliency map. The acuity map is calculated taking into considerations the characteristics of the human visual system, while the saliency map is calculated using theories from the field of visual attention. Our innovative solution takes multiple cameras inputs and creates a high-resolution panoramic video in real-time that can be streamed directly to the client. We parallelize the algorithm on a GPU to achieve a responsive interface and validate our results using user study. Our system accelerate graphics computation by a factor of 6 on a Google Cardboard display.