使用基於模擬退火的列組合加速 CNN 模型在 CPU 上的推論時間

Abstract

卷積神經網絡在各種電腦視覺問題中取得了相當好的成果。然而，最先進的卷積神經網絡模型的規模越來越巨大，這導致推論時間非常長和內存使用量很高。非結構化剪枝等模型壓縮技術可以在不影響準確性的情況下剪枝掉很大一部分參數，但有效利用其稀疏性仍然是一個挑戰。行組合通過將捲積濾波器矩陣中的多個稀疏行組合成單個密集行來壓縮非結構化剪枝後的卷積神經網絡模型。此外，在組合後行的每一列中剪枝除了最大幅度權重之外的所有權重，可以有效地進一步壓縮矩陣。然而，之前的論文沒有解決分割稀疏行的細節，以盡量減少額外剪枝對模型性能的負面影響。在這項工作中，我們首先證明行分區問題是一個 NP-Complete 問題。接下來，我們提出了一種基於模擬退火和全局非結構化剪枝的行組合方案，以最大限度地減少額外剪枝對模型性能的不利影響。我們在沒有特殊硬件支持的情況下使用 TVM AI 編譯器實現了行組合卷積神經網絡模型的加速。我們所提出的方案實現了更有效的模型壓縮，在 TinyImageNet 數據集稀疏度為 88% 的 VGG19 模型上精度提高 0.65%，推理時間加快 1.24 倍。

Convolutional Neural Networks (CNNs) have been successful in various computer vision tasks. However, the size of state-of-the-art CNN models tends to be tremendous, which results in very long inference times and high memory usage. Model compression technology such as unstructured pruning can prune a significant proportion of parameters without affecting accuracy, but efficient utilization of sparsity remains a challenge. Column combining compress unstructurally-pruned CNN models by combining multiple sparse columns in a convolutional filter matrix into a single dense column. In addition, pruning all but the largest magnitude weight in each row of the combined column further compresses the matrix effectively. However, previous work did not address the details of partitioning sparse columns to minimize the negative impact of additional pruning on the performance of the model. In this work, we first prove that the column partition problem is an NP-Complete problem. Next, we propose a column combining scheme based on simulated annealing and global unstructured pruning to minimize the adverse effects of additional pruning on model performance. We implement the acceleration of column-combined CNN models using the TVM AI compiler without special hardware support. The proposed scheme achieves more efficient model compression, leading to a 0.65% improvement in accuracy and a 1.24x faster inference time on VGG19 under 88% sparsity with the TinyImageNet dataset.