基於生成式雙流模型之行為辨識

Abstract

近年來，雙流式架構的神經網路在影片人類行為辨識的任務中展現了很強大的表現。雙流式網路的核心架構在於透過兩個子網路去抽取時間與空間的資訊，並透過這兩個資訊去做最後的行為辨識。然而，時間流的子網路依賴傳統的光流評估方法去抽取時間資訊，這是需要非常大量的運算資源以及對於儲存空間的要求也非常的高。為了解決這個問題，我們採用機器學習的技術去取代傳統的光流評估方法。在這篇論文，我們實現一個輕量化且低推論時間的雙流動作辨識模型。我們實現的光流評估模型利用對抗式網路的技術達到無監督式學習，我們的空間和時間子網路可以更進一步利用深度可分離卷積去減少模型的參數與運算複雜度，實驗結果顯示我們的方法達到即時的動作辨識並保有具有競爭力的結果。

Recently, the two-stream architecture of neural networks has shown strong performance for human action recognition in video tasks. The key idea of two-stream structure is to extract temporal information and spatial information from two sub-networks and fuses the information to recognize the actions. However, the temporal stream model relies on the traditional optical flow estimation methods to extract temporal information. It is computationally-expensive and storage-demanding. In order to address this problem, we use neural networks to replace traditional optical flow estimation methods. In this paper, we propose a light-weight and low inference time two-stream action recognition model. Our proposed optical flow estimation model achieves unsupervised learning by leveraging the techniques of Generative Adversarial Net (GAN). We can further reduce the number of parameters as well as computational complexity by using the depthwise separable convolution structure. The experimental results show that our method achieves real-time action recognition and retains competitive performance.