MAQEE：互適應量化與提早退出

Abstract

視覺Transformer（Vision Transformers, ViTs）透過自注意力機制在電腦視覺領域展現出色的表現，但由於其高昂的運算量，實際部署仍面臨相當挑戰。雖然混合精度量化（Mixed-Precision Quantization, MPQ）可降低模型容量，而提早退出（Early Exiting, EE）則能提升推論效率，然而將兩者整合時卻會面臨關鍵難題：一方面，量化雜訊會干擾提早退出的判斷穩定度；另一方面，動態的網路層使用情況會使得位元配置更加複雜。

為因應上述問題，我們提出 MAQEE（Mutual Adaptive Quantization with Early Exiting），提供一個可在量化與提早退出之間建立互惠關係的整合式框架。具體包含以下特色：

提早退出感知的混合精度量化（Early Exiting-Aware MPQ）：根據各層的實際使用狀況，動態調整並重新配置量化位元。量化後自我蒸餾（Post-Quantization Self-Distillation）：在量化後進行自我知識蒸餾，確保提早退出決策的穩定度。整合 SQNR 的量化風險控制（SQNR-incorporated Quantization-Aware Risk Control）：在量化過程中納入信號量化雜訊比（SQNR）指標，強化模型的風險控制能力。透過在 CIFAR-100 與 ImageNet-1K 上進行實驗，我們證實了 MAQEE 在維持 MPQ 與 EE 加速效率的同時，能比單純的 MPQ 基線模型提升最高可達 6% 的分類準確率，展現出 MAQEE 在實際應用中的效能與潛力。

Vision Transformers (ViTs) excel in computer vision through self-attention mechanisms but face deployment challenges due to high computational demands. While Mixed-Precision Quantization (MPQ) reduces model capacity and Early Exit-ing (EE) improves inference efficiency, their integration introduces critical challenges: quantization noise destabilizes exit decisions, while dynamic layer usage complicates bit allocation. We propose Mutual Adaptive Quantization with Early Exiting (MAQEE), a unified framework enabling mutual synergy between quantization and early exiting. Our approach features Early Exiting-Aware MPQ with layer utilization-based bit reallocation, Post-Quantization Self-Distillation for early exiting stability, and SQNR-incorporated Quantization-Aware Risk Control. Experiments on CIFAR-100 and ImageNet-1K demonstrate MAQEE’s effectiveness in achieving up to 6% higher classification accuracy compared with MPQ baseline while preserving the acceleration efficiency of both MPQ and EE.