利用簡化群最佳化對類神經網路、卷積神經網路和圖卷積網路進行超參數優化

Abstract

本研究探討了簡化群最佳化（SSO）和改進的簡化群最佳化（iSSO）在提升類神經網路（ANN）、卷積神經網路（CNN）和圖卷積網路（GCN）性能方面的應用。目前深度學習的文獻主要集中在網絡架構上；然而，本研究認為超參數優化—特別是激活函數、優化器、正則化技術和學習率調度器的優化—也扮演著至關重要的角色。為了證實這個假設，利用SSO來優化標準ANN模型、CNN模型LeNet和基本的GCN模型的超參數。實驗評估包括對ANN使用Iris、UCI ML手寫數字和Olivetti人臉數據集，對CNN使用MNIST、Fashion-MNIST和CIFAR-10數據集，對GCN使用Cora、Citeseer和Pubmed數據集。結果顯示，使用SSO及其改進版本iSSO的模型準確性均超過了原始配置。此外，iSSO能在相對較短的時間內找到最佳超參數。另外，SSO還提供了一種系統化的方法來選擇激活函數和優化器。

This research investigates the application of Simplified Swarm Optimization (SSO) and improved Simplified Swarm Optimization (iSSO) to enhance the performance of Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN) and Graph Convolutional Networks (GCN). Current literature in deep learning predominantly concentrates on network architectures; however, this study argues that hyperparameter optimization – specifically of the activation function, optimizer, regularization techniques, and learning rate schedulers – plays a critical role. To substantiate this hypothesis, SSO and iSSO were utilized to optimize hyperparameters in the standard ANN model, the CNN model LeNet, and a basic GCN model. The experimental evaluation included datasets such as Iris, UCI ML hand-written digits, and Olivetti faces datasets for ANNs, MNIST, Fashion-MNIST, and CIFAR-10 datasets for CNNs, and Cora, Citeseer, and Pubmed datasets for GCNs. The results indicated that the accuracy of models employing both SSO and its improved variant, iSSO, exceeded that of the original configurations. Moreover, the optimal hyperparameters were identified in a relatively short time in iSSO. Additionally, SSO provided a systematic methodology for the selection of activation functions and optimizers.