利用三維影像結合骨架追蹤和特徵姿勢搜尋以進行即時動作辨識

Abstract

在這篇論文中，我們提出一個迭代的方式結合彩色和深度資訊進行使用者動作以及姿勢辨識。不同於以往利用事先定義好的姿勢進行人體骨架初始化，我們利用基於深度的影像梯度統計特徵(HOG)建造一個特徵姿勢資料庫，當使用者進入相機視野後，我們便自動搜尋此資料庫尋找最適合的起始骨架，然後我們利用粒子濾波器(Particle Filter)結合2D和3D的特徵進行人體上半身骨架追蹤。同時，我們將追蹤的骨架資訊當作隱藏式馬可夫模型(Hidden Markov Model)的輸入值進行線上動作切割和辨識，為了提升辨識準確性和達到線上動作切割，我們修改了隱藏式馬可夫模型中的機率計算過程，並利用其機率的變化決定動作的開始和結束，此外我們也利用動作辨識的結果及特徵姿勢資料庫來矯正錯誤的骨架追蹤。最後，我們透過實驗來驗證此系統的整體效能和可靠性。

In this thesis, we propose an iterative approach which can not only recognize human actions but also estimate human upper body pose by combining color and depth information. Instead of using predefined pose to initialize human skeleton, we construct a key pose database with depth HOG feature as searching index. When user enters the camera view, we automatically search the database to get the initial skeleton. Then we use multiple importance sampling particle filter to track human upper body parts with 2D and 3D features as evidences. At the same time, we feed the tracking joints into the hidden Markov models (HMM) to on-line spot and recognize the performed action. In order to increase the recognition accuracy and perform on-line spotting, we modify the probability calculating process in HMM and propose an action spotting scheme based on the gradient of HMM probability. Besides, we apply the action recognition results and reuse our key poses database to rectify tracking error. To validate the effectiveness of the proposed action recognition approach, extensive experiments have been performed, of which the results appear to be quite promising.