基於向量量化之卷積神經網路處理器架構設計

Abstract

深度神經網路在許多邊緣端的電腦視覺任務上已經展現了令人印象深刻的表現，使得在手機上或是物聯網裝置上的深度神經網路加速器需求越來越多。然而，巨量的能量消耗與儲存量的需求使得硬體設計越來越困難。因此，在這篇論文中，我們提出了一個基於壓縮技術(向量量化)來同時減少的神經網路模型的大小與計算量的神經網路加速器。此外，我們設計了一種特化的處理單元與資料流，前者有不同的靜態隨機存取記憶體配置，後者則是可以使加速器支援不同的卷積濾波器的大小，並在輸入或輸出的維度極小時亦保持高度的使用率。與現今最佳的神經網路加速器相比，我們提出的加速器可以減少3.94倍的動態隨機存取記憶體存取量以及在單批次的神經網路推論下減少1.2倍的時間。

Deep neural networks (DNNs) have demonstrated impressive performance in many edge computer vision tasks, causing the increasing demand for DNN accelerator on mobile and internet of things (IoT) devices. However, the massive power consumption and storage requirement make the hardware design challenging. In this paper, we introduce a DNN accelerator based on a model compression technique vector quantization (VQ), which can reduce the network model size and computation cost simultaneously. Moreover, a specialized processing element (PE) is designed with various SRAM bank configurations as well as dataflows such that it can support different codebook/kernel sizes, and keep high utilization under small input or output channel numbers. Compared to the state-of-the-art, the proposed accelerator architecture achieves 3.94 times reduction in memory access and 1.2 times in latency for batch-one inference.