使用神經正切核解釋模型重程式化

Abstract

模型重程式化 (MR) [Chen, 2024] 是一種能夠有效率地使用計算資源的微調方 法，用於在不修改預訓練模型的權重的情況下，將預訓練的模型重新用於解決新 任務。本文通過神經正切核 (NTK) 的視角探討 MR，旨在增強對這一機器學習技 術的理解和效能。通過深入研究 MR 的理論基礎，並利用 NTK 框架的見解，本研 究闡明了促成 MR 算法成功的核心機制。借助 NTK 理論，這篇論文提供了新的見 解和解釋，並且加深對模型重程式化的理解。

Model reprogramming (MR) [Chen, 2024] is a resource-efficient fine-tuning method for repurposing a pretrained machine learning model to solve new tasks without modifying the pretrained weights. This paper explores MR through the lens of the Neural Tangent Kernel (NTK), aiming to enhance the comprehension and efficacy of this machine learning technique. By delving into the theoretical underpinnings of MR and utilizing insights from the NTK framework, this research elucidates the core mechanisms that contribute to the success of MR algorithms. Drawing on NTK theory, this work offers novel insights and explanations to deepen understanding of the model reprogramming process.