基於時間步長分群技術之擴散模型後量化技術

Abstract

近期擴散模型（Diffusion models）因卓越的 成像能力，使其在影像生成及編輯等領域獲得了廣泛的認可，並在許多方面超越了傳統模型的方法。儘管它們表現出色，這些模型固有的龐大運算量和記憶體需求往往限制了它們在資源受限的環境以及可攜式裝備的實際應用。為了因應這些挑戰，後訓練量化（ Post-Training Quantization, PTQ）成為了一種有效的解決方案。這個方法可以在不需要重新訓練的情況下減少所需的運算時間以及記憶體空間的需求。然而，由於擴散模型具備多重時間步長multiple-timestep 的特性，傳統的對稱後量化方法無法有效因應此特性往往造成明顯的量化誤差。 為了解決這一挑戰，我們提出了一項針對時間步長分群的量化方法，專門設計用於處理擴散模型中多個時間步長的特性 ，使得不同時間群組之間可以找到更適當的量化因子，進而提升量化的準確度。此外，我們發現SiLU層運算後的激活數值呈現非均勻分布，此分布可能導致劇烈的量化損失。為了減輕這一問題，我們引入了一種區域特定的量化策略，更準確地表示量化後的極值，從而提升模型的整體性能。通過整合這些創新方法，我們成功地開發出一種可應用於整體擴散模型的量化方法 。我們的實驗結果表明，該方法在進行8位量化後，仍能有效地保持 Frechet Inception Distance FID得分，突顯了此方法在實際應用中的潛力。

Diffusion models (DMs) have recently garnered widespread acclaim due to their superior imaging capabilities, which have demonstrated significant advancements over traditional methods. Despite their impressive performance, the extensive computational and memory demands inherent in these models often restrict their practical application on portable and resource-constrained devices. To mitigate these challenges, post-training quantization (PTQ) presents a promising solution that facilitates model compression and reduces runtime without necessitating retraining. However, traditional PTQ methods encounter substantial difficulties in addressing the unique time-variant distribution characteristics inherent in diffusion models. To address this challenge, we propose a novel timestep-grouping PTQ approach specifically designed to manage the complexities associated with multiple timesteps in diffusion models. Additionally, we have identified that non-uniform post-SiLU (Sigmoid Linear Unit) activations can result in significant quantization losses. To mitigate this issue, we introduce a region-specific quantization strategy that more accurately represents extreme values following quantization, thereby enhancing the model's overall performance. By integrating these innovative methods, we have successfully developed a fully quantized diffusion model that is feasible for hardware implementation. Our experimental results demonstrate that the proposed approach effectively maintains the Frechet Inception Distance (FID) score even after 8-bit quantization, underscoring the potential of our method for practical applications.