請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/91611
標題: | 深入探討使用聚合可逆神經網路之效能預測器於神經網路架構搜索的應用 Deep Dive into the Application of Aggregated Invertible Neural Network as Performance Predictor in Neural Architecture Search |
作者: | 楊大煒 Ta-Wei Yang |
指導教授: | 周承復 Cheng-Fu Chou |
關鍵字: | 神經網路架構搜索演算法,圖變分自編碼器,可逆神經網路,集成學習,機器學習, Neural Architecture Search,Graph Variational Autoencoder,Invertible Nueral Network,Ensemble Learning,Machine Learning, |
出版年 : | 2024 |
學位: | 博士 |
摘要: | 神經架構搜索(NAS)演算法旨在尋找效能表現最佳的模型,其中候選模型的評估通常是計算最昂貴的任務。NAS-Bench數據集有助於為架構評估訓練性能預測模型,進而能夠實現NAS搜索算法的發展,以找到最優的模型架構。在本論文中,我們把NAS問題轉換為神經網絡模型架構之效能預測的反問題。主要思想是使用圖形變分自編碼器(GVAEs)和可逆神經網絡(INNs)用來組成我們的效能預測模型InvertNAS,InvertNAS模型可以讓神經架構映射到其效能,反方向也可以。
這也就是說,GVAE能夠從取樣的神經架構中提取有用的低維潛在空間代碼,而聚合的INNs可以通過將這些潛在代碼映射到模型準確度來找出最終性能預測器。與其他NAS算法相比,結果顯示我們所提出的InvertNAS方法能夠於NASBench101上識別出排名第二的架構,在NASBench201上找到最優的架構,同時確保高查詢效率;我們相信,這種神經網絡架構之效能預測的反向對應方法是NAS的一個有前途與可用的解決方式。 Neural Architecture Search (NAS) algorithms could find the optimal or best-performing model, with candidate model evaluation typically being the most computationally expensive task. The NAS-Bench dataset facilitates the training of performance prediction models for architecture evaluation, enabling the development of NAS search algorithms to find optimal model architectures. In this thesis, we transform a NAS problem into an inverse problem of performance prediction for neural network architectures. The main idea is to use Graph Variational Autoencoders (GVAEs) and Invertible Neural Networks (INNs) to construct our performance prediction model, InvertNAS, which could map the neural architectures to their performance and vice versa. That is, Graph VAE is able to extract useful low-dimensional latent space codes from the sampling neural architectures, while the aggregation INNs could figure out the final performance predictor by mapping these latent codes to model accuracy. Compared with other NAS algorithms, the results indicate that our proposed method, InvertNAS, could identify the rank-two architecture on NASBench101 and the optimal architecture on NASBench201, all while ensuring high query efficiency; we believe that such an inverse method of neural network performance prediction is a promising solution for NAS. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/91611 |
DOI: | 10.6342/NTU202400317 |
全文授權: | 同意授權(限校園內公開) |
顯示於系所單位: | 資訊網路與多媒體研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-112-1.pdf 授權僅限NTU校內IP使用(校園外請利用VPN校外連線服務) | 7.39 MB | Adobe PDF | 檢視/開啟 |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。