應用於智慧物聯網聲音感測之高能效隱藏式神經網路處理器晶片

Abstract

物聯網聲音感測在智慧工廠與智慧家庭的應用日趨廣泛，為了提高異常偵測的準確度，深度神經網路也開始扮演判讀聲音訊號的重要角色。然而傳統的深度神經網路架構在載入權重和切換任務的過程中，需要大量高功耗之外部記憶體存取，也因此限制了將其部署於邊緣裝置上的能量效率。隱藏式神經網路提供一種更具能效的解決方案，其權重可以直接在晶片上產生，進而在維持準確度相當的情況下，大幅降低外部記憶體存取的資料量高達88.4%。除此之外，在使用相同的隨機權重下，只需要切換不同的遮罩，就能夠應用到不同的任務上，因此在記憶體有限的情況下，隱藏式神經網路可以有效率的支援不同聲音的偵測任務。本研究基於隱藏式神經網路的框架，提出第一個應用於異常聲音檢測之高能效神經網路處理器，此處理器藉由演算法與硬體架構之共同優化降低計算複雜度高達88.9%。本設計透過多層遮罩減少神經網路模型及其對應的遮罩大小，並且藉由零值計算省略以及時間和空間維度上的計算推測，進一步利用資料的稀疏性來降低神經網路推論過程中的運算量。本研究提出之處理器以40nm CMOS製程設計與製造，在1.85mm^2的核心面積上使用了1.69M個邏輯閘，在0.65-0.9V與20-200MHz下消耗5.6-45.7mW之功率。此晶片為首顆應用於物聯網聲音感測之隱藏式神經網路處理器。與中央處理器相比，在操作頻率低145倍的情況下，本晶片實現了11.8倍的加速，且功耗降低1.16×10^4倍。與過往文獻中的隱藏式神經網路處理器相比，本設計支援多位元權重之迴歸模型，並且透過多層遮罩縮小隨機網路模型，使其對應之遮罩可存於晶片上記憶體，不需要文獻所需之外部記憶體存取。本論文提出之晶片最高可達到88.1TOPS/W的能量效率，與過往文獻中的最佳設計相比，能量效率提升了30倍。

Artificial intelligence of things (AIoT)-based sound sensing has become increasingly prevalent in smart factories and smart homes. Deep neural networks (DNNs) are widely employed in sound-based monitoring systems to accurately and reliably detect potential malfunctions. However, conventional DNNs face notable limitations when deployed on edge devices, as loading weights and switching tasks require substantial external memory access (EMA), which consumes significant power. Hidden neural networks (HNNs) offer an energy-efficient solution to overcome these challenges. By utilizing on-chip generated random weights and binary masks to perform inference on a sparse subnetwork, HNNs reduce EMA by up to 88.4% with negligible accuracy loss. Unlike conventional DNNs, HNNs efficiently support detection tasks across multiple sound types using identical random weights with varying masks. This work presents the first NN processor for anomaly detection in IoT sound sensing, built on the framework of hidden neural networks. Energy efficiency is enhanced through algorithm-architecture co-optimization techniques, achieving up to an 88.9% reduction in computational complexity. A multicoated masking approach effectively reduces the network size and the corresponding mask size. Hierarchical bitmap encoding minimizes mask memory requirements, while a zero-skipping scheme and a space-time speculation technique exploit weight and activation sparsities in the HNN to significantly reduce computations. Fabricated in a 40nm CMOS technology, the chip integrates 1.69M logic gates in a core area of 1.85mm^2. It dissipates 5.6-to-45.7mW at a clock frequency of 20-to-200MHz from a 0.65-to-0.9V supply. Compared to the CPU implementation, the proposed HNN processor achieves an 11.8x speedup at a 145x lower clock frequency with 1.16×10^4x less power. In contrast to the previous HNN processor, this work supports regression models using multi-bit weights and utilizes multi-masks to effectively reduce network size, thereby eliminating the need for external memory. The proposed design delivers a maximum energy efficiency of 88.1TOPS/W, 30x higher than the state-of-the-art design.