使用稀疏立體空間類哈爾特徵之強健動態手勢識別系統

Abstract

手勢是一種有效而自然的人機交互方式。為了使計算機系統能夠以最貼近人類，最自然的方式來解讀手部的動作，我們採取以視覺為基礎來設計系統。最新的深度感知技術為手勢識別方法的發展提供的很好的機會。因此，在本篇論文中，我們提出了一種使用彩色及深度攝影機的強健動態連續手勢辨識系統。 現實的彩色和深度影像中，通常會存在雜訊和遮蔽的問題。為了從中自動識別出動態的手勢，我們提取了立體空間的類哈爾特徵來強健地表達手部複雜的空間訊息。利用類別分離度衡量方式，我們採用了一種新的特徵選擇方式來找出最具區分性的類哈爾特徵。接著我們還使用了稀疏表達的方式來對這些特徵進行編碼，藉此使系統更為準確穩定，尤其是在訓練資料相對有限的情況下。 稀疏的立體空間類哈爾特徵不但具有強健於雜訊和遮蔽的優點，而且可以通過使用自填充的積分體積來高效率地計算特徵值。這些關鍵的特徵顯著地提高了手勢分類和辨識的性能。在一個公共和一個自建的動態手勢資料庫上，我們都進行了實驗來評估我們的系統。相較於其他動態手勢識別方法，本篇論文的方法在準確性和穩定性上都表現出了優越性。

Hand gesture is an effective and natural way for human-robot interaction (HRI) and human-computer interaction (HCI). Vision-based system is chosen so that computer can understand hand activities naturally. Recent advances in depth sensing provide opportunities for development of approaches for hand gesture understanding. Therefore, in this thesis, we presents a robust dynamic hand gesture recognition system with an RGB-D sensor. In order to automatically recognize hand gesture from color and depth image sequences, where noise and occlusion are common problems, we extract steric Haar-like features to robustly represent the complicated spatial information of the hand. A novel feature selection approach, which takes the advantage of class separability measure, is employed to effectively ferret out the most discriminative features. We also use sparse coding method to encode these features so that it is less prone to over-fitting even when only limited amount of training data are available. Generally speaking, Sparse Steric Haar-like (SSH) features are efficient to compute by using the self-padding integral volume, in addition to the advantage of robustness to noise and occlusion. These crucial features significantly improve the performance of tracking and classification. Experiments with a public dynamic hand gesture dataset and a self-built hand gesture dataset show the superiority of the proposed system compared with the state-of-the-art approaches.