棒球場景中的時序二維姿態辨識優化與應用於無監督三維人體動作辨識

Abstract

在科技日新月異的時代，運動員也能透過科技輔助使表現更上一層樓，本論文提出一套完整的無標記動作捕捉系統。分析動作能協助運動員追蹤自行表現的變化，無標記能避免影響運動員的表現，並能更廣泛的運用於各個場域。

此系統透過AlphaPose進行二維影像辨識，以及多視角立體電腦視覺進行三維重建。使用棋盤版以及立方體對相機模型預先校正，以獲得相機矩陣參數。最後利用生物力學軟體OpenSim，運算人體動作的關節角度。

配合真實球場架設的情況，相機偶爾會有些微移動，畫面偏移造成嚴重的三維誤差。此時可以透過最小平方法修正外部參數矩陣，不需前往球場重新進行校正。並且分析實際球場之校正誤差，在一公尺約佔180像素之影像大小中，投手活動範圍中的校正最大誤差約為10毫米。而攝影機對視可能造成較攝影機夾角90度的10倍誤差量。

另外，夜間比賽的辨識效果不佳，造成部分外群值影響整體動作分析，本論文提出多個時序性的方法去除部分離群值，時間序列局部最大值透過影片的連續性改善AlphaPose的熱圖取點，關節配對法可以交換左右辨識相反的關節，自動異常偵測法自動尋找異常處並用一階擬合的方式改善辨識錯誤區間。最佳的演算法配合可使關節位置平均誤差達到30.31毫米。透過OpenSim能相當的穩定動作，分析手肘以及膝蓋角度分別有8.13度、5.15度的角度誤差。

In this era of rapid technological advancement, athletes can enhance their performance with the assistance of technology. This thesis presents a markerless motion capture system. Players can track performance changes without being encumbered by markers, enabling broader applications across various fields.

The system utilizes AlphaPose for 2-D image recognition and multi-view stereo computer vision for 3-D reconstruction. Camera models are pre-calibrated using a checkerboard pattern and a cube to obtain camera matrix parameters. Finally, joint angles of human movement are computed using the biomechanics software OpenSim.

In real-world scenarios, cameras may experience slight movements, leading to significant 3-D errors due to image shifts. The external parameter matrices can be corrected using least squares methods without recalibration at the field. Analysis of calibration errors in actual stadiums reveals a maximum calibration error of approximately 10 mm within an image size of 180 pixels per meter. Opposite views for cameras can result in error quantities up to 10 times greater than those at camera angles of 90 degrees.

Moreover, poor recognition performance during nighttime matches can introduce outliers affecting overall motion analysis. This thesis proposes several temporal methods to remove outliers, including leveraging video continuity for improving AlphaPose heatmap estimating, Matching Joints to swap incorrectly identified joints, Auto-detect Anomaly Range finding outliers, and using first-order fitting to remove recognition error intervals. Combining these algorithms achieves an average joint position error of 30.31 mm. OpenSim ensures stable motion analysis, with elbow and knee angle errors of 8.13 degrees and 5.15 degrees, respectively.