請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/71550
標題: | 基於一次性超網之深度神經網路推薦系統的效能評估 An Efficient Performance Estimation on Deep Neural Network Recommendation System with One-Shot SuperNet |
作者: | Chu-Siang Huang 黃楚翔 |
指導教授: | 洪士灝(Shih-Hao Hung) |
關鍵字: | 神經網路架構搜索,效能估計,遷移學習, Neural Architecture Search,Performance Estimation,Transfer Learning, |
出版年 : | 2020 |
學位: | 碩士 |
摘要: | 近年來,許多邊緣計算平台已經引入了深度學習操作。然而,深度神經網絡模型的訓練仍然需要大量計算,並且在數據集龐大時需要雲服務和硬件加速器。此外,可以通過基於雲的神經體系結構搜索服務針對單一邊緣設備優化深度神經網絡模型,但對於多目標而言,計算成本可能會高得令人望而卻步,且自從數據集和設計以來,隱私問題就引起了關注。使用者必須向雲服務提供商披露邊緣設備相關資訊。在本文中,我們提出了一種有效的深度神經網絡推薦系統來解決上述挑戰性問題。首先,我們採用一種先前提出的方法,即一次性超網,以減少多目標神經結構搜索的計算成本。接著,我們提出使用端到端性能預測指標來解決隱私問題,這些指標僅要求用戶提供某些採樣網絡體系結構的評估結果。我們利用遷移學習技術將數據集的特徵和硬件規格轉移到性能預測器中,以提高性能評估的效率,而不從用戶那裡獲取數據集和要求硬體規格。實驗表明,我們的方法只需要不到十分之一的樣本就可以實現相同水平的推理延遲性能預測,而只需要五分之一的樣本就可以預測圖像分類基準中的前一個類別的精確度。 Recently, many edge computing platforms have been introduced to perform deep learning operations near the users. However, training for deep neural network models remains to be computationally intensive and requires cloud services and hardware accelerators when the dataset is huge. Moreover, deep neural network models can be optimized for individual edge devices by cloud-based neural architecture search (NAS) services, the computational cost can be prohibitively high for multi-objective NAS, and privacy concerns have raised since the datasets as well as the design of the edge devices have to be revealed to the cloud service providers. In this thesis, we propose an efficient deep neural network recommendation system to address the aforementioned challenging issues. First, we adopt a previously proposed method, one-shot supernet, to reduce the computational cost for multi-objective NAS. Then we propose to address privacy concerns with end-to-end performance predictors, which only require users to provide the evaluation results for certain sampled network architectures. Instead of acquiring datasets and demanding hardware specifications from the users, we leverage the transfer learning technique to transfer the characteristics of datasets and hardware specifications into our performance predictors to improve the efficiency of performance estimation. Experiments show that our method only needs less than one tenth of samples to achieve the same level of performance prediction for inference latency and one fifth samples for predicting the top-1 accuracy in image classification benchmarks. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/71550 |
DOI: | 10.6342/NTU202004336 |
全文授權: | 有償授權 |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
U0001-1211202023222600.pdf 目前未授權公開取用 | 8.57 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。