基於最佳化與訊息濾波器之兩階段式三維人體姿態估測系統

Abstract

隨著人機互動需求的逐年上升，三維人體姿態估測在近年來更為受到了重視。為了使機器人、工具機械乃至於電子裝置能夠感知、分析，進而學習、理解，最後隨人類肢體動作自動反應，正是需要一個快速且足夠準確的估測方法。 基於視覺的估測方式雖然沒有達到使用目標標記的光學式方法的估測準確度，但是僅使用單一鏡頭相機帶來低成本與容易建置的優點，也因此被認為是更為適合用於現實應用的方法。隨著電腦運算量能的提升與機器學習技術的發展，已經有許多採用深度學習技術的基於視覺的估測方法被提出、研究。然而三維的估測不像二維的估測已經達到精準的即時運算，三維的估測仍然無法解決運算速度與估測準確度的兩難問題。而三維估測的結果合理性與重新投影回二維影像的重投影誤差也還不是很受到重視，前者在需要人體動態分析的應用中會是一個重要的關鍵，後者則是與估測方法的一般化可行性有相關。 本篇論文中提出了一個運算快速且容易建置的多視角三維人體姿態估測方法，此方法利用不同視角的多台單一鏡相機，然後使用從相機影像得到的二維人體估測作為輸入資料以確保計算速度，並通過一個兩階段式的估測系統與最後的後處理得到平滑的連續三維的人體姿態估測。在第一階段的系統中會進行最佳化運算，並將上述提及的重投影誤差納入考量，一定程度保證此方法的一般化可行性。在第二階段的系統中引入了基於運動模型的濾波器，將基本運動學的條件、關係包含在濾波器的模型中，讓估測的姿態符合基本運動學。更進一步來說，我們所提出的兩階段式系統架構將兩種估測方法組合在一起，能夠達到各自優缺點的互補，將非線性及線性的條件簡單的融入在姿態估測中。然後，為了得到更平滑的連續估測動態，我們在系統之後再加入一段資料後處理。最後，此方法估測結果的合理性也會以運動學與幾何學的角度下在此篇論中進行分析、探討。

The techniques of 3D human pose estimation have gain much attention in recent years due to the growing demand of human robot or human machine interactions. In order to sense, analyze, learn, realize, and eventually react automatically to human action, a fast and adequately accurate 3D human pose estimation method is much needed for all robots,machines, and electronic devices. Despite not being as accurate as those target-marking optoelectronic methods, vision-based methods using only monocular cameras has the advantage of low cost and easy implementation, and thus, is considered as the method more likely to be applied to real-world applications. With the increasing computation power and the well-developed machine learning techniques, many vision-based estimation methods based on deep learning have been proposed and studied. But, unlike the 2D estimations achieving real-time computation with good precision, 3D pose estimations remain struggling between computation speed and estimation accuracy. Moreover, the rationality of estimated motion and the reprojection error back to the 2D iamges have not gain much attention, which the former is an crucial part of applications requiring motion analysis, and the latter is related to the generalizability of the estimation methods. A multi-view 3D human pose estimation method using multiple monocular cameras is proposed in this thesis for easy implementation and fast computation by applying optimization through multi-view real-time 2D estimations and model-based filtering. In the first stage of the system, the reprojection error of the estimates are taken into consideration within the optimization process to assure the generalizability of the method to some extent. Then, the basic kinematic relations of the human body are also included in the dynamic model of the model-based filtering, which is the second stage of the system. With the two-stage structure of the system, the disadventages of each stage are covered up by the other stage, and the nonlinear and linear relations and constraints are easily included within the estimation results. Next, a postprocess method is applied to further smooth the motion of the acquired estimates after the two-stage estimation. Last but not least, the rationality of the estimated motion is also analyzed in this thesis in respect of the values of kinematic and geometric parameters.