基於位移導向之立體影像視角合成應用於單一或多視角彩色深度相機

Abstract

With the rapid improvement of technology, RGB/D cameras have been growing more and more popular. Depth map plays an important role for many 3D applications in human–computer interaction systems. There are several technical challenges in producing a high-quality synthesized view. To generate more comfortable novel view has become the bottleneck for current research. Multimedia applications such as 3DTV, and virtual reality provide viewers with 3D experience by presenting videos from different viewpoints to our eyes. Through we can build rich 3D maps of environments is an important task for mobile robotics, with applications in navigation, manipulation, semantic mapping, and telepresence. But 3D point clouds frame-to-frame alignment and dense 3D reconstruction require high bandwidth, memory and computational costs because of costly iterative operations, the original point cloud is computationally expensive for real-time system implementation. View synthesis is just like 3D information projection to 2D, is an efficient implementation in daily life. Most popular view synthesis system adopted by the ITU/MPEG standard uses depth image based rendering (DIBR) techniques. In this thesis, we propose to tackle the artifacts, pinhole, disocclusion of RGB-D multiview images when synthesizing new views of a scene by changing its viewpoint. We first examine how ghost contour come from, why the disocclusion region cannot be seen in the original view but exposed in the virtual view, and pinholes/Cracks appear in the derived frame for surfaces whose normal has rotated towards the user in the derived frame. 2D information from images transformation to 3D and make connection between reference view and novel view. We analyze 3D warping techniques: background erosion is proposed here to remove the wrongly warped boundary, forward warping is to write a single derived pixel for each warped reference pixel. And we define a vector displacement as corresponding feature points’ “movement” between different views to help us to do backward warping. Further more, we point out the disocclusion is the area between foreground-background edge displacement difference, and we combine convention inpainting technique and our growing guidance to get improvement on hole filling. Large holes are pretty hard to be filled with an acceptable subjective quality. The situation can be relieved by multiview. Despite of z-buffering, we also propose view weighting based on the distance between reference and novel view and the other method winner take all. In order to generate free view, we exploit quaternion rotation to do inter-view interpolation, and analyze the quality versus view point. Finally, we can provide quality good and comfortable virtual view synthesis through our proposed.