透過姿態估計與特徵學習實現太極拳輔助學習系統的實時反饋

Abstract

24式太極拳是傳統楊氏太極拳的簡化版本，它在保留了太極核心動作的同時，減少了招式的複雜性，使其更容易學習，適合作為全民健身運動來推廣。而對於初學者而言，跟隨教學影片中教練的動作來學習太極招式是最簡便的方式，然而影片並不會提供任何反饋，使學習者難以得知自己的動作的正確性。

隨著人工智慧的發展和姿態估計模型的逐漸成熟，現在可以從一般的網路攝影機拍攝的影像，推估出影像中人們的骨架關節點資訊，並進一步利用這些資訊對人的動作進行分析，然而想評估學習者的動作是否和教練相符，需要太極拳的專業知識並手工設計相似度計算方法。為此，我們收集大量太極動作影片並將其轉成骨架資訊，利用圖卷積神經網路模型，以資料去驅動模型學習太極動作的特徵，將一個動作(Motion)轉為一個具該動作特徵的向量表示(Embedding)，並以三元組損失函數(Triplet Loss)優化這些向量表示，使相似的動作向量更加接近，不相似的動作向量更加遠離。通過這種方法，我們便能簡單的利用餘弦相似性去評估兩個動作向量間的相似性，並作為評分，即時的反饋給學習者。實驗結果顯示，使用模型輸出的動作向量來計算動作相似度，能夠有效地辨別相似動作和相異動作。相較於直接比較動作的骨架關節點坐標位置，我們的方法能提升至多24%的辨別準確率，提供更為穩定且明確的評分。

為了使模型易於使用，我們將模型與教學影片整合成一個界面，使學習者在跟隨教練動作的同時，可以實時看到自己當前動作的評級。在練習完招式後，學習者能檢視各評級中自己與教練動作的差異，從而改善動作，提升訓練效果。

The 24-form Tai Chi is a simplified version of traditional Yang-style Tai Chi Chuan. By retaining the essential movements while reducing the complexity of the techniques, it is easier to learn and suitable for promoting as a fitness exercise for the general public. For beginners, learning Tai Chi Chuan by following instructional videos is the most accessible way. However, these videos do not provide any feedback, making it difficult for learners to know if their movements are correct.

With advancements in Artificial Intelligence and pose estimation, it is now possible to estimate the skeleton joint data of people in videos captured by standard webcams. This data can be used to analyze human motions. However, evaluating whether the learner's motions match the instructor's requires domain knowledge in Tai Chi Chuan and effort to manually design similarity evaluation methods. To address this challenge, we created a motion dataset of 24-form Tai Chi and used a graph convolutional network to learn the features of Tai Chi movements from the data. The model converts skeleton motion data into motion embeddings that capture the motion features. Trained with a triplet loss function, it ensures that the embeddings of similar motions are closer together, while those of dissimilar motions are further apart. This allows us to simply use cosine similarity to evaluate the similarity between two motion embeddings and provide real-time feedback to learners. Experimental results show that evaluating similarity using our motion embeddings can effectively differentiate similar and dissimilar motions, with an accuracy improvement of up to 24% over joint coordinate comparisons, offering clearer and more consistent similarity scores.

For user-friendly purposes, we integrated our model with instructional videos into an interface that allows users to see real-time feedback on their movements while following the instructor. After practice, users can review the differences between their movements and the instructor's within each rating range, enabling them to make improvements and enhance their practice effectiveness.