結合分數階高斯濾波器與剪枝之深度神經網路壓縮方法

Abstract

隨著深度神經網路在實際應用中的廣泛使用，神經網路壓縮技術的重要性日益提升，特別是在資源受限的邊緣設備上。因為在現實世界的應用中，深度神經網路（DNN）往往對其造成沉重的硬體負擔。

儘管已有諸多方法致力於壓縮模型參數，部署這些模型在邊緣設備上仍面臨諸多挑戰。為了應對這個問題，我們提出了結合分數階高斯濾波器和剪枝的壓縮架構FGFP（Fractional Gaussian Filter and Pruning）。該框架整合了分數階微分和高斯函數以構建分數階高斯濾波器（FGFs）。為了降低分數階微分運算的計算複雜度，我們引入了Grünwald–Letnikov分數階導數來近似分數階微分方程式。每個FGF捲積核的參數量被精簡至僅剩七個。進一步地，為了優化運算效率，我們透過運算順序重排，在分數高斯濾波器架構中實現了卷積運算計算量減少64.06%。除分數高斯濾波器架構外，我們的FGFP架構還整合了自適應非結構化剪枝（AUP）以實現更高的壓縮率。

在各種架構和基準數據集上的實驗表明，我們的FGFP架構在準確性和壓縮率方面都優於最近的方法。在CIFAR-10上，ResNet-20在模型尺寸縮小85.2%的情況下，準確度僅下降了1.52%。在ILSVRC2012上，ResNet-50在模型尺寸縮小69.1%的情況下，準確度僅下降了1.63%。

為了評估該框架的實際適用性，我們在消費級筆記型電腦上部署了該模型進行推理。實驗結果顯示，該方法在端到端推理過程中實現了3.33倍的速度提升。

Neural network compression techniques are becoming increasingly important nowadays because Deep Neural Networks (DNNs) impose heavy hardware resource loads on edge devices in real-world applications.

While many methods compress neural network parameters, deploying these models on edge devices remains challenging. To address this, we propose the Fractional Gaussian Filter and Pruning (FGFP) framework, which integrates fractional-order differential calculus and Gaussian function to construct fractional Gaussian filters (FGFs). To reduce the computational complexity of fractional-order differential operations, we introduce Grünwald-Letnikov fractional derivatives to approximate the fractional-order differential equation. The number of parameters for each kernel in FGF is minimized to only seven. Furthermore, through operation reordering, we achieve a 64.06% reduction in the computational load of convolution operations within the fractional Gaussian filter architecture. Beyond the architecture of fractional Gaussian filters, our FGFP framework also incorporates Adaptive Unstructured Pruning (AUP) to achieve higher compression ratios.

Experiments on various architectures and benchmarks show that our FGFP framework outperforms recent methods in accuracy and compression. On CIFAR-10, ResNet-20 achieves only a 1.52% drop in accuracy while reducing the model size by 85.2%. On ILSVRC2012, ResNet-50 achieves only a 1.63% drop in accuracy while reducing the model size by 69.1%.

To assess the framework’s real-world applicability, we deployed the model on a consumer laptop, achieving a 3.33× end-to-end speedup.