基於線性區間分析之混合式濾波器剪枝法

Abstract

近年來，模型壓縮技術已成為將大型模型部署於資源受限系統中的重要步驟。本研究提出了一種基於線性區間分析的混合式濾波器剪枝方法，通過設置距離閥值和範數閥值，將叢集式剪枝法和範數剪枝法的特性結合起來。此外，利用神經網路架構搜尋領域中的線性區間分析法，可以快速判斷模型架構的鑑別度特性。透過以上設計，混合式濾波器剪枝法可以在短時間內搜尋出剪枝後表現最佳的結構和對應閥值，從而達到壓縮模型的效果。使用此方法可以在 CIFAR10 資料集上將 ResNet56 網路之浮點數運算量 (FLOPs) 壓縮至原本的 20%，同時僅有不到2% 的精確度損失，在其他資料集與網路架構上也可以在高壓縮率下達到較低的精確度損失。

Model compression techniques have become crucial for deploying large models on resource-limited systems. This study proposes a hybrid filter pruning method based on linear region analysis. Our approach combines the strengths of cluster pruning and norm-based filter pruning by introducing thresholds based on Euclidean distance and norm distance. Moreover, we incorporate the linear region analysis method from neural network architecture search to estimate the performance of trained model architectures. This enables us to efficiently search for the optimal pruned structure with corresponding threshold values, achieving effective model compression. Experimental results on the CIFAR10 dataset with ResNet56 demonstrate that our hybrid pruning method can achieve an 80% reduction in FLOPs with less than 2% accuracy loss. Furthermore, our method consistently performs well in terms of accuracy drop at high compression rates across various datasets and network architectures.