以骨頭長度修正增強三維人體骨架預測

Abstract

現今在三維人體骨架預測的研究，主要集中於預測三維關節座標，而忽視了其他重要的物理限制，例如骨頭長度的一致性以及人體的對稱性。我們提出了骨頭長度的預測模型，模型使用循環神經網路的架構，捕捉全面的影片資訊，以達到準確的預測。為了使訓練更有效，我們合成了符合物理限制的骨頭長度資料，並提出了全新的資料增強方法。此外，我們提出了骨頭長度校正，在保持骨頭轉向的狀態下，把骨頭長度替換成我們的預測值。結果顯示，在經過骨頭長度校正後，現存的三維人體骨架預測模型都能有顯著的改善。我們更進一步使用預測出的骨頭長度，對人體骨架預測模型進行微調，也同樣能有很好的改善。我們的骨頭長度預測模型超越了過去的最佳結果，並且在Human3.6M資料集的多個評估方法上，校正與模型微調的方法都能有效地改善。

Current approaches to 3D human pose estimation primarily focus on regressing 3D joint locations, often neglecting critical physical constraints such as bone length consistency and body symmetry. This work introduces a recurrent neural network architecture designed to capture holistic information across entire video sequences, enabling accurate prediction of bone lengths. To enhance training effectiveness, we propose a novel augmentation strategy using synthetic bone lengths that adhere to physical constraints. Moreover, we present a bone length adjustment method that preserves bone orientations while substituting bone lengths with predicted values. Our results demonstrate that existing 3D human pose estimation models can be significantly enhanced through this adjustment process. Furthermore, we fine-tune human pose estimation models using inferred bone lengths, observing notable improvements. Our bone length prediction model surpasses the previous best results, and our adjustment and fine-tuning method enhance performance across several metrics on the Human3.6M dataset.