深度神經網絡的機器反學習

Abstract

本論文介紹了一個高效的計算優化框架 ECO，它將 CP 算法（最初由 Cauwenberghs & Poggio (2000)[4] 提出）適應於深度神經網絡（DNN）模型中的精確卸載。ECO 利用單一的模型架構，結合了基於 DNN 的特徵轉換函數和 CP 算法，實現了精確的數據刪除，而無需完全重新訓練模型。我們證明了 ECO 不僅提高了效率，還保持了原始基礎 DNN 模型的性能，令人驚訝的是，它甚至在效果上超越了機器反學習領域裡的黃金標準——重新訓練。最重要的是，我們為第一個調適 CP 算法原本設計用於進行逐一排除評估的遞減學習，以在 DNN 模型中實現精確遺忘，徹底移除特定數據資料的影響。我們計劃開源我們的程式碼，以促進此方法在機器遺忘領域的進一步研究。

This paper introduces ECO, an efficient computational optimization framework that adapts the CP algorithm—originally proposed by Cauwenberghs & Poggio (2000)[4]—for exact unlearning within deep neural network (DNN) models. ECO utilizes a single model architecture that integrates a DNN-based feature transformation function with the CP algorithm, facilitating precise data removal without necessitating full model retraining. We demonstrate that ECO not only boosts efficiency but also maintains the performance of the original base DNN model, and surprisingly, it even surpasses naive retraining in effectiveness. Crucially, we are the first to adapt the CP algorithm’s decremental learning for leave-one-out evaluation to achieve exact unlearning in DNN models by fully removing a specific data instance's influence.