基於三維手勢之多層式遠端電視控制技術

Abstract

本論文提出一套基於三維手勢控制之多階層控制模式智慧型電視系統。 在整體架構上，我們提出融合自然使用者介面(Natural User Interface)和圖形使用者介面(Graphic User Interface)的操作模式，並且依據此兩種介面的性質設計不同的功能和目的和操作方法。 本系統所採用的手勢樣式為九種自然且直覺的手勢，根據其操作性質可分成五種自然使用者介面(Natural User Interface)以及四種圖形使用者介面(Graphic User Interface)；我們在系統設計上使用多階層控制模式之架構，其中每種控制模式皆對應至電視的一種操控型態且各自擁有獨特手勢操作功能，此外也可藉由特定手勢控制在不同模式間切換。 對於五種自然使用者介面之手勢辨識，本論文蒐集使用者操作這些手勢常見的表達方式，藉由觀察分析這些手勢資料，我們提出混合型手勢辨識演算法，用以辨識手勢以及解決誤判的問題，並將手勢定義成使用者能輕鬆直覺操作的方法，由實驗結果可得本系統達到高度的手勢辨識準確度與只有少量的誤檢率。 在圖形使用者介面的模式中，本論文致力於解決在進行點擊(click)動作時偏移的問題，我們提出一套融合三維軌跡辨識與手部有限狀態機的演算法：對於人類的手在空間不同位置下習慣會出現的軌跡蒐集分析並且使用高斯模型進行三維空間下的點擊軌跡辨識，配合手部有限狀態機和使用者介面的組合，我們比通用演算法(只看深度變化值)提升非常多的準確率。

This thesis presents a multi-mode remote control method which allows the user to interact with a Smart TV by switching between four different modes: Standby Mode, TV Watch Mode, TV Control Mode, and Cursor Mode. Our system allows the users to switch among different control modes through the predefined gestures. Among all gesture recognition approaches, we are especially interested in the geometric trajectory-based template matching approaches, which distinguish different gestures by using trajectory patterns. That is to say, those approaches are used to concentrate on recognizing isolated gesture trajectory. However, in practical case, the gesture sequence is a continuous stream of unknown length, and unknown start and end point. More importantly, some different gestures may contain similar trajectories, which are very difficult to be recognized. This paper presents a 3D gesture recognition approach, which is designed to discriminate gestures with similar palm trajectories. Some experiments have been performed to evaluate the accuracy of our 3D gesture recognition system. In Cursor Mode, we propose a freehand click gesture recognition approach by using palm trajectory. General approaches are used to recognize click gesture through detecting a straightforward press movement. However, the users usually do not press perfectly straight, so those approaches may fail to detect click gesture. Here, we named this issue as “misaligned click problem.” Unlike the general click recognition approaches may suffer misaligned click problem, our approach learns the 3D palm trajectories in locations within available click region and using our click-gesture control finite state machine to control click progress. In our thesis, some experiments have been performed to evaluate the accuracy of our 3D gesture recognition system. We have compared 3D gesture our recognition system against four recognizers: Our algorithm with elbow information, Protractor (2D xy-projection template matching approach), Protractor (2D xz-projection template matching approach), Protractor3D (3D trajectory matching approach). Experimental results on self-collected action database demonstrated that our proposed approach can successfully achieve higher recognition accuracy and lower false positive rate. On the other hand, to evaluate our freehand click gesture recognizer, we tested our approach on a self-collected click dataset, and compared it with general click approach. Experimental results show that our click recognition approach achieves higher recognition accuracy than the general approach. Keywords: multi-mode remote control; Gesture recognition; Natural User Interface; Graphic User Interface; Click gesture recognition; Misaligned click problem.