綜觀模型架構之代表性： 基於濾波器重要性學習之注意力機制通道剪枝

Abstract

通道剪枝（Channel pruning）為一模型壓縮的方法以加速卷積神經網路 (Convolutional neural networks, CNNs)。然而，近期研究決定結構重要性時並無考慮特徵之冗餘性，以及濾波器於接續數層卷積層之遞減影響。因此本文提出一個兩階段剪枝框架，DOMINOS。在不損失過多準確率下，首先衡量結構重要性以有效移除冗餘模型結構。確立輕量化的模型架構後，DOMINOS 進一步使用注意力機制加以強調富含預測資訊的結構。本文定義 feature representativeness 用以衡量特徵的獨特性，以及定義 domino influences 用以表示濾波器於接續卷積神經網路層逐層衰減的影響，並使用存活分析當中的韋伯機率模型衡量該影響。此外，本文設計了一個 channel-to-block 剪枝方法有效移除模型當中不重要的塊（blocks）以解決連結層輸出特徵之通道數不一致的問題。實驗結果顯示 CNN 模型基於本文所提出的剪枝框架，在不失過多準確率情況下，大幅減少浮點數運算量以及參數量，其效果顯著超出近期相關研究之成果。

Channel pruning is a model compression approach to accelerate convolutional neural networks (CNNs). Nevertheless, previous approaches determined the structure’s importance without considering the redundancy of features and the decaying influences of filters on the subsequent layers. In this thesis, therefore, we propose a two-stage pruning framework, DOMINOS, to first effectively remove redundant structures without a significant accuracy loss by leveraging the structure's importance. After identifying the lightweight architecture, DOMINOS applies the attention mechanism to focus on the informative structures for prediction. We define feature representativeness to measure the uniqueness of feature patterns and domino influences to represent the decaying influences of filters on the subsequent CNN layers by leveraging Weibull probabilistic models in survival analysis. Moreover, we devise a channel-to-block pruning approach to efficiently remove unimportant blocks to solve the mismatch of channel sizes for output feature maps in linked layers. Experimental results manifest that CNN models under DOMINOS pruning framework achieve prominent reductions in flops and parameters without compromising the accuracy, significantly outperforming the state-of-the-art.