非獨立同分布之影像資料使用於半監督分散式學習的效能優化

Abstract

現代的移動終端裝置，例如:手機和行車記錄器，無時無刻都在收 集大量的資料，而這些資料對於很多物聯網和人工智慧的應用是很有 價值的。然而隨著對於隱私問題的日漸重視，使用者不願意將他們的 資料上傳到伺服器做進一步的分析，這牽涉到深度模型的訓練。因此 Google 提出的聯邦學習在不需要回傳資料的情況下，利用分散式學習 的方式訓練一個深度學習模型。然而在現實環境下，終端裝置上的資 料皆為非獨立通分布，會造成訓練時引起權重分歧並最終導致續練出 來的模型性能降低。在這篇論文中，我們提出一種創新的聯邦學習方 案，並設計了一個名為聯邦交換(FedSwap)的新操作，在聯邦訓練期 間替換一些原本的聯邦平均(FedAvg)的操作，以減輕權重分歧所造 成的影響。我們將我們的方法實作圖像分類在 CIFAR-10 標準資料上 和物體偵測在現實世界影像資料上。實驗結果顯示，圖像分類的準確 率提高了 3.8%，物體偵測可以提高 1.1%。

Modern mobile end devices, such as smartphones and dash cams, are collecting numerous data that can be useful for various IoT and AI appli- cations. However, as the concern of privacy rises, users are not willing to provide their data to the server for further data analysis, which involves deep learning model training. Therefore, Google proposes Federated Learning, of- fering distributed end devices a way to train a deep learning model without passing their private data to a centralized server. However, in the real world circumstances, the data on the end devices are non-independent and identi- cally distributed (non-IID) such that it may cause weight divergence during training and eventually result in a considerable decrease in the model perfor- mance. In this thesis, we propose an innovative Federated learning scheme, in which we design a new operation called Federated Swapping (FedSwap) to replace some Federated Averaging (FedAvg) operations based on a few shared data during federated training to alleviate the adverse impact of weight divergence. We implement our method on both image classification using CIFAR-10 benchmark data and object detection using the real world video data. Experiment results show that the accuracy of image classification is increased by 3.8%, and the object detection task can be improved by 1.1%.