請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/71550完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 洪士灝(Shih-Hao Hung) | |
| dc.contributor.author | Chu-Siang Huang | en |
| dc.contributor.author | 黃楚翔 | zh_TW |
| dc.date.accessioned | 2021-06-17T06:03:02Z | - |
| dc.date.available | 2025-11-06 | |
| dc.date.copyright | 2020-12-25 | |
| dc.date.issued | 2020 | |
| dc.date.submitted | 2020-11-20 | |
| dc.identifier.citation | [1] Anonymous. ResPerfNet: Deep Residual Learning for Regressional Performance Modeling of Deep Neural Networks. Sept. 2020. [2] B. Baker, O. Gupta, N. Naik, and R. Raskar. Designing Neural Network Architectures using Reinforcement Learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. [3] M. Berman, L. Pishchulin, N. Xu, M. B. Blaschko, and G. Medioni. AOWS: Adaptive and optimal network width search with latency constraints. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11217–11226, 2020. [4] L. Bossard, M. Guillaumin, and L. Van Gool. Food-101–mining discriminative components with random forests. In European conference on computer vision, pages 446–461. Springer, 2014. [5] H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han. Once-for-All: Train One Network and Specialize it for Efficient Deployment. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. [6] H. Cai, L. Zhu, and S. Han. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. [7] T. Chau, \. Dudziak, M. S. Abdelfattah, R. Lee, H. Kim, and N. D. Lane. Brp-nas: Prediction-based nas using gcns. arXiv preprint arXiv:2007.08668, 2020. [8] Cheng-Yueh Liu. Co-designing Artificial Intelligence and High-performance Computing Systems. PhD Thesis, National Taiwan University, Jan. 2019. [9] G. Chu, O. Arikan, G. Bender, W. Wang, A. Brighton, P.-J. Kindermans, H. Liu, B.Akin, S. Gupta, and A. Howard. Discovering Multi-Hardware Mobile Models via Architecture Search. arXiv:2008.08178 [cs], Aug. 2020. arXiv: 2008.08178. [10] X. Chu, B. Zhang, and R. Xu. MoGA: Searching beyond MobileNetV3. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4042–4046. IEEE, 2020. [11] A. Coates, A. Ng, and H. Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223, 2011. [12] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le. AutoAugment: Learning Augmentation Strategies From Data. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 113–123, Long Beach, CA, USA, June 2019. IEEE. [13] X. Dai, A. Wan, P. Zhang, B. Wu, Z. He, Z. Wei, K. Chen, Y. Tian, M. Yu, and P. Vajda. FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function. arXiv preprint arXiv:2006.02049, 2020. [14] X. Dai, P. Zhang, B. Wu, H. Yin, F. Sun, Y. Wang, M. Dukhan, Y. Hu, Y. Wu, andY. Jia. Chamnet: Towards efficient network design through platform-aware model adaptation. In Proceedings of the IEEE Conference on computer vision and pattern recognition, pages 11398–11407, 2019. [15] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, June 2009. ISSN: 1063-6919. [16] T. DeVries and G. W. Taylor. Improved Regularization of Convolutional Neural Networks with Cutout. arXiv:1708.04552 [cs], Nov. 2017. arXiv: 1708.04552. [17] J.-D. Dong, A.-C. Cheng, D.-C. Juan, W. Wei, and M. Sun. Dpp-net: Device-aware progressive search for pareto-optimal neural architectures. In Proceedings of the European Conference on Computer Vision (ECCV), pages 517–531, 2018. [18] X. Dong, M. Tan, A. W. Yu, D. Peng, B. Gabrys, and Q. V. Le. AutoHAS: Efficient Hyperparameter and Architecture Search. arXiv:2006.03656 [cs], Oct. 2020. arXiv: 2006.03656. [19] Z. Guo, X. Zhang, H. Mu, W. Heng, Z. Liu, Y. Wei, and J. Sun. Single path one-shot neural architecture search with uniform sampling. arXiv preprint arXiv:1904.00420, 2019. [20] S. Gupta and B. Akin. Accelerator-aware Neural Network Design using AutoML. arXiv preprint arXiv:2003.02838, 2020. [21] A. Hard, K. Rao, R. Mathews, S. Ramaswamy, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, and D. Ramage. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604, 2018. [22] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. [23] J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018. [24] Y. Hu, X. Wu, and R. He. Tf-nas: Rethinking three search freedoms of latency-constrained differentiable neural architecture search. arXiv preprint arXiv:2008.05314, 2020. [25] D. Justus, J. Brennan, S. Bonner, and A. S. McGough. Predicting the computational cost of deep learning models. In 2018 IEEE International Conference on Big Data (Big Data), pages 3873–3882. IEEE, 2018. [26] S. Kornblith, J. Shlens, and Q. V. Le. Do better imagenet models transfer better?In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2661–2671, 2019. [27] J. Krause, M. Stark, J. Deng, and L. Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pages 554–561, 2013. [28] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. 2009. Publisher: Citeseer. [29] G. Larsson, M. Maire, and G. Shakhnarovich. FractalNet: Ultra-deep neural networks without residuals. In ICLR, 2017. [30] X. Li, Y. Zhou, Z. Pan, and J. Feng. Partial Order Pruning: For Best Speed/Accuracy Trade-Off in Neural Architecture Search. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9137–9145, Long Beach, CA, USA, June 2019. IEEE. [31] H. Liu, K. Simonyan, and Y. Yang. DARTS: Differentiable Architecture Search. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. [32] Z. Lu, K. Deb, E. Goodman, W. Banzhaf, and V. N. Boddeti. Nsganetv2: Evolutionary multi-objective surrogate-assisted neural architecture search. arXiv preprint arXiv:2007.10396, 2020. [33] Z. Lu, G. Sreekumar, E. Goodman, W. Banzhaf, K. Deb, and V. N. Boddeti. Neural Architecture Transfer. arXiv:2005.05859 [cs], May 2020. arXiv: 2005.05859. [34] J. Mei, Y. Li, X. Lian, X. Jin, L. Yang, A. L. Yuille, and J. Yang. AtomNAS: Fine-Grained End-to-End Neural Architecture Search. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. [35] X. Mo and J. Xu. Energy-Efficient Federated Edge Learning with Joint Communication and Computation Design. arXiv:2003.00199 [cs, eess, math], Feb. 2020. arXiv: 2003.00199. [36] S. J. Pan and Q. Yang. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345–1359, Oct. 2010. Conference Name: IEEE Transactions on Knowledge and Data Engineering. [37] H. Pham, M. Guan, B. Zoph, Q. Le, and J. Dean. Efficient Neural Architecture Search via Parameters Sharing. In International Conference on Machine Learning, pages 4095–4104. PMLR, July 2018. ISSN: 2640-3498. [38] P. Ramachandran, B. Zoph, and Q. V. Le. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017. [39] E. Real, A. Aggarwal, Y. Huang, and Q. V. Le. Regularized Evolution for Image Classifier Architecture Search. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):4780–4789, July 2019. tex.ids: realRegularizedEvolutionImage2019a number: 01. [40] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018. [41] D. Stamoulis, R. Ding, D. Wang, D. Lymberopoulos, B. Priyantha, J. Liu, and D. Marculescu. Single-path nas: Device-aware efficient convnet design. arXiv preprint arXiv:1905.04159, 2019. [42] M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, and Q. V. Le. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2820–2828, 2019. [43] M. Tan and Q. V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946, 2019. [44] A. Wan, X. Dai, P. Zhang, Z. He, Y. Tian, S. Xie, B. Wu, M. Yu, T. Xu, and K. Chen. Fbnetv2: Differentiable neural architecture search for spatial and channel dimensions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12965–12974, 2020. [45] T. Wang, K. Wang, H. Cai, J. Lin, Z. Liu, H. Wang, Y. Lin, and S. Han. APQ: Joint Search for Network Architecture, Pruning and Quantization Policy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2078–2087, 2020. [46] B. Wu, X. Dai, P. Zhang, Y. Wang, F. Sun, Y. Wu, Y. Tian, P. Vajda, Y. Jia, and K. Keutzer. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10734–10742, 2019. [47] Y. Xiong, H. Liu, S. Gupta, B. Akin, G. Bender, P.-J. Kindermans, M. Tan, V. Singh, and B. Chen. MobileDets: Searching for Object Detection Architectures for Mobile Accelerators. arXiv preprint arXiv:2004.14525, 2020. [48] Y. Xu, L. Xie, X. Zhang, X. Chen, B. Shi, Q. Tian, and H. Xiong. Latency-Aware Differentiable Neural Architecture Search. arXiv:2001.06392 [cs], Mar. 2020. arXiv: 2001.06392. [49] Q. Yang, Y. Liu, T. Chen, and Y. Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1–19, 2019. Publisher: ACM New York, NY, USA. [50] Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu. Federated learning. Synthesis lectures on artificial intelligence and machine learning. Morgan Claypool Publishers, 2019. [51] J. Yu and T. Huang. AutoSlim: Towards One-Shot Architecture Search for Channel Numbers. arXiv:1903.11728 [cs], May 2019. arXiv: 1903.11728. [52] B. Zoph and Q. V. Le. Neural Architecture Search with Reinforcement Learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. [53] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le. Learning Transferable Architectures for Scalable Image Recognition. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8697–8710, June 2018. ISSN: 2575-7075. | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/71550 | - |
| dc.description.abstract | 近年來，許多邊緣計算平台已經引入了深度學習操作。然而，深度神經網絡模型的訓練仍然需要大量計算，並且在數據集龐大時需要雲服務和硬件加速器。此外，可以通過基於雲的神經體系結構搜索服務針對單一邊緣設備優化深度神經網絡模型，但對於多目標而言，計算成本可能會高得令人望而卻步，且自從數據集和設計以來，隱私問題就引起了關注。使用者必須向雲服務提供商披露邊緣設備相關資訊。在本文中，我們提出了一種有效的深度神經網絡推薦系統來解決上述挑戰性問題。首先，我們採用一種先前提出的方法，即一次性超網，以減少多目標神經結構搜索的計算成本。接著，我們提出使用端到端性能預測指標來解決隱私問題，這些指標僅要求用戶提供某些採樣網絡體系結構的評估結果。我們利用遷移學習技術將數據集的特徵和硬件規格轉移到性能預測器中，以提高性能評估的效率，而不從用戶那裡獲取數據集和要求硬體規格。實驗表明，我們的方法只需要不到十分之一的樣本就可以實現相同水平的推理延遲性能預測，而只需要五分之一的樣本就可以預測圖像分類基準中的前一個類別的精確度。 | zh_TW |
| dc.description.abstract | Recently, many edge computing platforms have been introduced to perform deep learning operations near the users. However, training for deep neural network models remains to be computationally intensive and requires cloud services and hardware accelerators when the dataset is huge. Moreover, deep neural network models can be optimized for individual edge devices by cloud-based neural architecture search (NAS) services, the computational cost can be prohibitively high for multi-objective NAS, and privacy concerns have raised since the datasets as well as the design of the edge devices have to be revealed to the cloud service providers. In this thesis, we propose an efficient deep neural network recommendation system to address the aforementioned challenging issues. First, we adopt a previously proposed method, one-shot supernet, to reduce the computational cost for multi-objective NAS. Then we propose to address privacy concerns with end-to-end performance predictors, which only require users to provide the evaluation results for certain sampled network architectures. Instead of acquiring datasets and demanding hardware specifications from the users, we leverage the transfer learning technique to transfer the characteristics of datasets and hardware specifications into our performance predictors to improve the efficiency of performance estimation. Experiments show that our method only needs less than one tenth of samples to achieve the same level of performance prediction for inference latency and one fifth samples for predicting the top-1 accuracy in image classification benchmarks. | en |
| dc.description.provenance | Made available in DSpace on 2021-06-17T06:03:02Z (GMT). No. of bitstreams: 1 U0001-1211202023222600.pdf: 8776439 bytes, checksum: 03af473c5b96b545a33028b09f2b6a9c (MD5) Previous issue date: 2020 | en |
| dc.description.tableofcontents | 誌謝 i 摘要 ii Abstract iii 1 Introduction 1 2 Background and Related Work 4 2.1 Neural Architecture Search 4 2.1.1 Early Works 4 2.1.2 Platform-Aware NAS 5 2.1.3 One-Shot Supernet Method 5 2.2 Latency Estimation 6 2.3 Transfer Learning 7 3 Methodology 8 3.1 One-Shot Supernet Optimization 8 3.2 Performance Predictor 9 3.3 Performance Transferring 9 4 Experiment 13 4.1 Overall Experimental Setup 13 4.1.1 Hardware Platforms 13 4.1.2 Benchmark dataset 14 4.1.3 Evaluation Criteria 14 4.2 Latency Predictor 15 4.2.1 Experimental Setup 15 4.2.2 Experiment Results 15 4.3 Accuracy Predictor 20 4.3.1 Experimental Setup 20 4.3.2 Experiment Results 21 4.4 Efficiency of the Proposed Recommendation System 22 5 Conclusion and Future Work 25 Appendices 25 A Case Study: FLOPs versus Latency 26 Bibliography 27 | |
| dc.language.iso | en | |
| dc.subject | 遷移學習 | zh_TW |
| dc.subject | 神經網路架構搜索 | zh_TW |
| dc.subject | 效能估計 | zh_TW |
| dc.subject | Neural Architecture Search | en |
| dc.subject | Performance Estimation | en |
| dc.subject | Transfer Learning | en |
| dc.title | 基於一次性超網之深度神經網路推薦系統的效能評估 | zh_TW |
| dc.title | An Efficient Performance Estimation on Deep Neural Network Recommendation System with One-Shot SuperNet | en |
| dc.type | Thesis | |
| dc.date.schoolyear | 109-1 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 施吉昇(Chi-Sheng Shih),徐慰中(Wei-Chung Hsu),梁文耀(William W.-Y. Liang) | |
| dc.subject.keyword | 神經網路架構搜索,效能估計,遷移學習, | zh_TW |
| dc.subject.keyword | Neural Architecture Search,Performance Estimation,Transfer Learning, | en |
| dc.relation.page | 33 | |
| dc.identifier.doi | 10.6342/NTU202004336 | |
| dc.rights.note | 有償授權 | |
| dc.date.accepted | 2020-11-20 | |
| dc.contributor.author-college | 電機資訊學院 | zh_TW |
| dc.contributor.author-dept | 資訊工程學研究所 | zh_TW |
| 顯示於系所單位： | 資訊工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| U0001-1211202023222600.pdf 未授權公開取用 | 8.57 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。