基於交叉驗證及分散式集成聯邦學習之架構

Abstract

本研究先探討聯邦學習框架之侷限性、傳統分散式聯邦學習與分散式架構矛盾之處及侷限性、其造成之問題及困境，接著了解並分析數個嘗試在聯邦學習領域引入集成學習技術之研究；發掘其受聯邦學習框架制約並限制創新之關鍵，最後提出更高彈性、高容錯、高可擴展性之架構：基於交叉驗證及分散式集成聯邦學習之架構(Architecture Based on Cross-validation and Decentralized Ensemble Federated Learning, ABCDEFL)。另對於在分類任務中之集成學習，本論文亦提出一個更細緻的集成輸出方法：Classwise Weighted Majority Voting (CWMV)。 其後本論文以一系列實驗驗證對ABCDEFL各方面可能性和優勢之猜想；以及CWMV概念之效果，最終確認了所有有疑慮之猜想；展示及討論了ABCDEFL之各種可能性和優勢，並證實了CWMV輸出集成方法相較於傳統集成方法在各種情況中皆更有優勢。實驗程式碼開源，但論文發布當下毫無可讀性，需後續修繕。 論文最後根據本研究提出數個未來可研究之方向或重要之新概念；並於最後一節由研究中所得發想、思考及討論前往通用人工智慧、人工意識之可預見障礙、可能道路及其潛在威脅、應對方案等相關看法。

The research first investigates the limitations in the framework of Federated Learning (FL), the limitations in the framework of conventional Decentralized Federated Learning (DFL), the apparently visible contradictions between the common design of DFL and the strengths of decentralized architecture, and the predicaments caused by all the above. Then, we consult recent researches trying to adopt Ensemble Learning (EL) into the narrow framework of FL to spot the key restrictions that depressed their innovation. Finally, an architecture that is more flexible, robust, and scalable is proposed: Architecture Based on Cross-validation and Decentralized Ensemble Federated Learning (ABCDEFL). Also, in the domain of classification problems, a more meticulous output aggregation method for EL is proposed: Classwise Weighted Majority Voting (CWMV). A series of experiments are designed to verify the expected strengths of proposed methods, and the results confirm them. Thus, the thesis shows and argues the possibilities and advantages brought by ABCDEFL, and the superiority of CWMV is validated. The experiment scripts are open-sourced, yet without feasible readability at the publication of this thesis. They should be revised in the future. In the final chapter, several important innovative concepts and directions for future researches are proposed. In the last section, inspired by the gains from this research, we also embark on discussions about the expected handicaps toward Artificial General Intelligence or even Artificial Consciousness, a possible road through them, the potential threats awaiting at the destination, and my proposed way to counter.