使用顯著軟標籤的動態混合以應對長尾骨架動作識別

Abstract

基於骨架資料的動作辨識由於其計算效率高以及在動態環境中的穩定表現，近年來逐漸受到關注。然而，現有大部分方法仍依賴於平衡的資料集，即每個類別都擁有相近數量的樣本，卻忽視了現實生活中的資料分佈往往是不均衡的現象，這樣的長尾分佈問題會顯著降低模型的效能。本論文針對長尾分佈的骨架動作辨識提出了新的解決方案，旨在強化代表性不足的樣本類別的表現，從而克服資料不平衡導致的模型偏差問題。 為此，我們提出了一種結合動態混合技術與新型軟標籤計算的資料增強方法，專注於提升尾部類別的辨識準確率。我們的方法包含一個專為少數類別設計的選擇策略，並透過新型軟標籤計算，確保混合樣本與其對應混合標籤之間的關聯性，進一步提高增強資料的品質。 我們在長尾的 NTU RGB+D 資料集、N-UCLA 資料集，以及我們實驗室收集的長尾 AIMS 嬰兒動作資料集上進行了實驗，結果顯示我們的方法在整體動作辨識表現上優於當前最先進的方法，特別是在尾部類別的辨識上取得了顯著的提升，展現了其應對現實世界資料集的潛力。

Skeleton-based action recognition has gained increasing attention in recent years due to its computational efficiency and robust performance in dynamic environments. However, most existing methods still rely on balanced datasets, where each class has a similar number of samples, ignoring the fact that data in real-world scenarios is often imbalanced. This long-tailed distribution significantly reduces the model’s effectiveness. In this paper, we propose a novel solution to address the challenge of long-tailed skeleton-based action recognition by enhancing the performance of underrepresented classes, thereby mitigating the bias introduced by data imbalance. To this end, we introduce a data augmentation approach that combines dynamic mixup techniques with a new soft label calculation method, specifically focused on improving the recognition accuracy of tailed classes. Our method includes a selection strategy designed for minority classes, and the soft label calculation ensures the correlation between mixed samples and their corresponding mixed labels, further enhancing the quality of the augmented data. We conducted experiments on the long-tailed NTU RGB+D dataset, N-UCLA dataset, and the long-tailed AIMS infant action dataset collected in our lab. The results demonstrate that our method outperforms state-of-the-art approaches in overall action recognition performance, with particularly significant improvements in the recognition of tailed classes, demonstrating its potential in tackling real-world datasets.