FedUA:一種適用於圖像分類的可感知不確定性的蒸餾式聯邦學習方案

Abstract

近年來，聯邦式學習（Federated Learning）逐漸變成資訊領域重要的研究課題。而聯邦式學習強調學習任務是由鬆散的設備（或稱為用戶端）合力共同解決，此情境下存在的重要挑戰包括：用戶端間的資料是不平衡且非獨立同分布的 (Non-IID)，並且設備間的溝通受限於有限的傳輸頻寬而不可靠。上述議題對於聯邦式學習是棘手的挑戰。本文從深度神經網路的不確定性切入聯邦式學習的效能評估並提出新的模型聚合架構（Model Aggregation）。此架構是以知識蒸餾（Knowledge Distillation）為基礎佐以量化深度神經網路的不確定性（Uncertainty in DNN）相關評估方法，進而提升學習效果。實驗在圖像分類（Image Classification）的任務上，證實我們提出的模型聚合架構可有效的解決非獨立同分布的問題，尤其在限制傳輸成本的狀態下，有不錯的表現。

In recent years, Federated Learning has gradually become an important research topic in the field of information theory. Federated learning emphasizes that learning tasks are jointly solved by loose devices (or clients). Important challenges in this situation include: the data among clients is unbalanced and non-IID, and the communication between devices is unreliable due to limited transmission bandwidth. The above issues are intractable to federated learning. This paper starts from the uncertainty of deep neural network to evaluate the effectiveness of federated learning and proposes a new model aggregation architecture. This scheme is based on knowledge distillation and quantifies the uncertainty in DNN related evaluation methods of deep neural networks to improve the learning performance. The experiments on the task of image classification confirm that our proposed model aggregation scheme can effectively solve the problem of non-IID data distribution, especially when the transmission cost is limited, and it has a good performance.