語言引導之變換器於聯邦式多標籤分類

Abstract

聯邦學習(FL)是一種新興的模型學習框架,該方法使多個用戶能夠在不共享私人數據的情況下協作訓練一個強大的模型,以保護隱私。大多數現有的FL方法僅考慮傳統的單標籤圖像分類問題,忽略了將任務轉移到多標籤圖像分類時的影響。然而,在現實世界的FL場景中,FL仍然難以應對用戶在本地數據分佈上的異質性,而在多標籤圖像分類中,這個問題變得更加嚴重。有鑑於變換器模型於中央化學習設定下之成功經驗,我們於是提出了一個新穎的FL框架用於多標籤分類。由於在訓練過程中本地客戶可能觀察到部分標籤之間的相關性,直接聚合本地更新的模型將不能產生滿意的全局模型效能。因此,我們提出了一個新穎的FL框架「語言引導之變換器」(FedLGT)來應對這一具有挑戰性的任務,旨在利用、傳輸不同客戶間的知識以學習一個強大的全局模型。根據我們於各種多標籤數據集(例如 FLAIR,MS-COCO等)進行大量實驗,我們展示了FedLGT能夠達到足夠好的性能,在多標籤FL場景下顯著優於標準FL技術。

Federated Learning (FL) is an emerging paradigm that enables multiple users to collaboratively train a robust model in a privacy-preserving manner without sharing their private data. Most existing approaches of FL only consider traditional single-label image classification, ignoring the impact when transferring the task to multi-label image classification. Nevertheless, it is still challenging for FL to deal with user heterogeneity in their local data distribution in the real-world FL scenario, and this issue becomes even more severe in multi-label image classification. Inspired by the recent success of Transformers in centralized settings, we propose a novel FL framework for multi-label classification. Since partial label correlation may be observed by local clients during training, direct aggregation of locally updated models would not produce satisfactory performances. Thus, we propose a novel FL framework of Language-Guided Transformer (FedLGT) to tackle this challenging task, which aims to exploit and transfer knowledge across different clients for learning a robust global model. Through extensive experiments on various multi-label datasets (e.g., FLAIR, MS-COCO, etc.), we show that our FedLGT is able to achieve satisfactory performance and outperforms standard FL techniques under multi-label FL scenarios. Code is available at https://github.com/Jack24658735/FedLGT.