以手勢控制實時生成現場物體三維模型之互動應用裝置

Abstract

This paper presents a novel interaction for Handheld Augmented Reality, Grab-Carry-Release, to allow users to communicate visually and experiment with the placement of physical objects in a real scene in situ through smartphones. In the Grab-Carry-Release process, a user first grabs a physical object on the touchscreen of a smartphone to reconstruct a virtual copy of the object on the fly, then carries it by moving the smartphone, and finally releases it at the target location. To achieve this, a RGB-D camera is attached to the smartphone in order to capture the geometry and appearance of the object simultaneously with a contour tracking and model reconstruction pipeline. The RGB-D camera is also used for locating the reconstructed virtual model with its surrounding physical environment. In addition, possible usage scenarios for in-situ AR interaction and in-situ & remote mobile AR collaboration are developed. Finally, a user study is conducted to gather user feedbacks for future improvement of the proposed interactions.