Sparse ReRAM Engine: 聯合探索壓縮神經網路之權重與激活稀疏性

Abstract

利用神經網路模型的稀疏性以減少無效的計算為普遍使用的方法以達到高能效的深度神經網路推論加速器。然而由於緊密耦合的縱橫式結構，在基於可變電阻式記憶體之神經網路加速器下探索稀疏性尚為較少關注的部分。現有的可變電阻式記憶體之神經網路加速器架構研究假設整個縱橫式陣列可以在單一周期內啟動。 然而考慮推論之精確度，矩陣-向量計算在實踐中必須以更小的粒度執行，稱之為操作單位(Operation Unit)。基於OU的架構創造了新的機會來探索深度神經網路的稀疏性。在本論文中，我們提出了第一個實際的稀疏可變電阻式記憶體引擎(Sparse ReRAM Engine)同時利用權重與激活的稀疏性。我們的評估顯示提出的方法可以有效的消除無效的計算，並且提供可觀的效能改善與能源節省。

Exploiting model sparsity to reduce ineffectual computation is a commonly used approach to achieve energy efficiency for DNN inference accelerators. However, due to the tightly coupled crossbar structure, exploiting sparsity for ReRAM-based NN accelerator is a less explored area. Existing architectural studies on ReRAM-based NN accelerators assume that an entire crossbar array can be activated in a single cycle. However, due to inference accuracy considerations, matrix-vector computation must be conducted in a smaller granularity in practice, called Operation Unit (OU). An OU-based architecture creates a new opportunity to exploit DNN sparsity. In this paper, we propose the first practical Sparse ReRAM Engine that exploits both weight and activation sparsity. Our evaluation shows that the proposed method is effective in eliminating ineffectual computation, and delivers significant performance improvement and energy savings.