跨域知識遷移學習於人體骨架動作辨識

Abstract

人體動作基於人體骨架之識別(Skeleton­based Action Recognition)目前有許多的資料集。但是每一個資料集之間存在著非常多的相異之處。像是拍攝方向、不同的人體關鍵點、以及不同種類的行為等等。我們通常都是直接讓同一個資料集當作訓練及測試集，因此常常不會用到其他資料集的知識。為了解決這個問題，我們提出了一個跨域知識遷移學習基於梯度翻轉層(Gradient Reversal Layer)與圖形卷積網路的模型來有效的轉換並利用不同資料集來的知識來影響其他資料集的結果。根據NTU­RGB+D 60遷移到其他資料集的實驗，我們提出的方法可以讓這些其他資料集的表現大幅的增加準確度並且超過目前只做在目標資料集基於時空圖型卷積網路的最佳演算法，並且證明我們提出的方法的影響力。

For skeleton­-based action recognition, there are many different datasets; however, since there also exist many differences between skeleton action datasets, including view­points, the number of available joints for a skeleton, the type of actions, etc, we can only train an individual model for each dataset respectively and cannot effectively leverage the knowledge from one dataset to another. To address this issue, we propose a cross­domainknowledge transfer module based on gradient reversal layer for graph convolutional net­work to effectively transfer the knowledge from one domain to another. With extensive experiments from NTU­RGB+D 60 to other datasets, the proposed approach achieves significantly improved results using different state­of­the­art spatio­temporal graph con­volutional networks as compared with those trained on the target dataset only, and this also demonstrates the effectiveness of the proposed approach.