基於雙重排名機制的深度學習模型軟混合濾波器剪枝

Abstract

過往的剪枝技術大多使用網路中的單一結構來進行評估，如卷積層(convolutional layer)或批量標準化層(batch-normalization layer)，來決定剪枝目標。然而，這種方法沒辦法有效利用每個層中所有結構，因此效果有限。為了能更 加全面的考慮每個層中的各種結構，我們提出雙重排名軟混合濾波器剪枝 (Soft Hybrid Filter Pruning using a Dual Ranking Approach, DR-SHFP)，該方法建立在軟剪枝(Soft Filter Pruning, SFP)的基礎上，並引入了雙重排名的方法。DR-SHFP使用了一個排名機制，並給每個濾波器一個排名，這個排名是由卷積層和批量標準化層共同決定的。通過同時評估這兩個層，我們的方法能夠更全面地捕捉來自層結構的信息，打破了單一結構評估的限制。因此，DR-SHFP 能夠更有效地識別和選擇要進行剪枝的濾波器，從而提高性能。實驗結果表明，DR-SHFP在CIFAR-10、CIFAR-100和Tiny-ImageNet等資料集上具有優異的性能。

Conventional pruning techniques typically focus on evaluating a single structure in the network, such as the convolutional layer or batch normalization layer, to identify pruning targets. However, this approach fails to effectively leverage the potential of all structures within each layer of the network. In order to comprehensively consider the various structures in each layer, we propose a novel method called Soft Hybrid Filter Pruning using a Dual Ranking Approach (DR-SHFP), which builds upon Soft Filter Pruning (SFP) by introducing a dual-ranking approach. DR-SHFP incorporates a ranking system that assigns a rank to each filter in a collaborative manner, taking into account both convolutional layers and batch normalization layers. By simultaneously evaluating both types of layers, our method captures more information from the layer structures, surpassing the limitations of single-structure evaluation. Consequently, DR-SHFP can identify and select filters more effectively for pruning, leading to improved performance. Experimental results demonstrate the effectiveness of DR-SHFP on benchmark datasets such as CIFAR-10, CIFAR-100 and Tiny-ImageNet. The proposed method outperforms other soft pruning methods, showcasing its capability to achieve excellent performance in various settings.