請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/72329
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 楊佳玲(Chia-Lin Yang) | |
dc.contributor.author | Zhi-Lin Ke | en |
dc.contributor.author | 柯志霖 | zh_TW |
dc.date.accessioned | 2021-06-17T06:35:45Z | - |
dc.date.available | 2020-08-24 | |
dc.date.copyright | 2018-08-24 | |
dc.date.issued | 2018 | |
dc.date.submitted | 2018-08-15 | |
dc.identifier.citation | [1] cgroups. https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt.
[2] Geforce GTX 1070. https://www.nvidia.com/en-us/geforce/products/10series/ geforce-gtx-1070/. [3] Tensorflow API r1.4. https://www.tensorflow.org/versions/r1.4/api_docs/. [4] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow. org. [5] A. Abdiansah and R. Wardoyo. Time complexity analysis of support vector machines (svm) in libsvm. International Journal Computer and Application, 2015. [6] Y. Bengio. Practical recommendations for gradient-based training of deep architectures. In Neural networks: Tricks of the trade, pages 437–478. Springer, 2012. [7] L. Bottou. Curiously fast convergence of some stochastic gradient descent algorithms. In Proceedings of the symposium on learning and data science, Paris, 2009. [8] K.-W. Chang and D. Roth. Selective block minimization for faster convergence of limited memory large-scale linear models. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 699–707. ACM, 2011. [9] S. Chetlur, C. Woolley, P. Vandermersch, J. Cohen, J. Tran, B. Catanzaro, and E. Shelhamer. cudnn: Efficient primitives for deep learning. arXiv preprint arXiv:1410.0759, 2014. [10] J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, A. Senior, P. Tucker, K. Yang, Q. V. Le, et al. Large scale distributed deep networks. In Advances in neural information processing systems, pages 1223–1231, 2012. [11] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large- Scale Hierarchical Image Database. In CVPR09, 2009. [12] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9:1871–1874, 2008. [13] I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org. [14] C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi, and S. Sundararajan. A dual coordinate descent method for large-scale linear svm. In Proceedings of the 25th international conference on Machine learning, pages 408–415. ACM, 2008. [15] C.-J. Hsieh, S. Si, and I. Dhillon. A divide-and-conquer solver for kernel support vector machines. In International Conference on Machine Learning, pages 566–574, 2014. [16] INTEL®. 3D XPOINT™ TECHNOLOGY. https://www.intelsalestraining.com/ infographics/memory/3DXPointc.pdf. [17] INTEL®. Optane™ SSD DC P4800X Series. http://www.intel.com/content/ www/us/en/solid-state-drives/optane-ssd-dc-p4800x-brief.html. [18] INTEL®. SSD 750 SERIES. https://www.intel.com/content/www/us/ en/products/memory-storage/solid-state-drives/gaming-enthusiast-ssds/ 750-series.html. [19] S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015. [20] A. Kayid, Y. Khaled, and M. Elmahdy. Performance of cpus/gpus for deep learning workloads. 05 2018. [21] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012. [22] M. Li, T. Zhang, Y. Chen, and A. J. Smola. Efficient mini-batch training for stochastic optimization. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 661–670. ACM, 2014. [23] M. Lichman. Uci machine learning repository: Higgs. Covertype, US Census, 1990. [24] S.-H. Lim, S. R. Young, and R. M. Patton. An analysis of image storage systems for scalable training of deep neural networks. system, 5(7):11, 2016. [25] Q. Meng, W. Chen, Y. Wang, Z.-M. Ma, and T.-Y. Liu. Convergence analysis of distributed stochastic gradient descent with shuffling. arXiv preprint arXiv:1709.10432, 2017. [26] G. Montavon, G. B. Orr, and K.-R. Müller. Tricks of the trade. 1998. [27] R. Raina, A. Madhavan, and A. Y. Ng. Large-scale deep unsupervised learning using graphics processors. In Proceedings of the 26th annual international conference on machine learning, pages 873–880. ACM, 2009. [28] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211–252, 2015. [29] T. Serafini and L. Zanni. On the working set selection in gradient projection-based decomposition techniques for support vector machines. Optimization Methods and Software, 20(4-5):583–596, 2005. [30] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229, 2013. [31] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. [32] D. Wang, D. Irani, and C. Pu. Evolutionary study of web spam: Webb spam corpus 2011 versus webb spam corpus 2006. In Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom), 2012 8th International Conference on, pages 40–49. IEEE, 2012. [33] WD. HDD-WD10EZEX. https://www.wdc.com/content/dam/wdc/website/ downloadable_assets/eng/spec_data_sheet/2879-771436.pdf. [34] H.-F. Yu, C.-J. Hsieh, K.-W. Chang, and C.-J. Lin. Large linear classification when data cannot fit in memory. ACM Transactions on Knowledge Discovery from Data (TKDD), 5(4):23, 2012. [35] H.-F. Yu, H.-Y. Lo, H.-P. Hsieh, J.-K. Lou, T. G. McKenzie, J.-W. Chou, P.-H. Chung, C.-H. Ho, C.-F. Chang, Y.-H. Wei, et al. Feature engineering and classifier ensemble for kdd cup 2010. In KDD Cup, 2010. [36] G.-X. Yuan, C.-H. Ho, and C.-J. Lin. An improved glmnet for l1-regularized logistic regression. Journal of Machine Learning Research, 13(Jun):1999–2030, 2012. | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/72329 | - |
dc.description.abstract | 支援向量機(Support Vector Machine, SVM)與深度神經網絡(Deep Neural Network, DNN)機器學習演算法在近幾年受到大家的關注。在 訓練機器學習演算法時,對所有訓練資料進行隨機洗牌(Random shuffling) 可以提高測試準確度(Testing accuracy) 與收斂速度(Convergence rate)。然而,由於硬式磁碟機(Hard disk drive, HDD)中的隨機存取 (Random access) 速度慢,在實際系統中實現訓練資料的隨機洗牌並不 是一個簡單的過程。為了避免頻繁地對硬式磁碟機的隨機存取,現有 的解決方法通常會限制隨機洗牌的效果。由於新興的基於非揮發性記 憶體的儲存裝置(Non-volatile memory-based storage) 提供快速的隨機存 取,例如的Intel Optane SSD,我們提出一個輕量級的隨機洗牌方法 LIRS,透過隨機洗亂整個訓練數據集的索引,並直接從儲存裝置中讀 取選定的訓練資料並組成批量(Batch) 以達到隨機洗牌的效果。實驗結 果顯示,採用LIRS 可以使SVM 和DNN 的總訓練時間平均減少49.9% 和43.5%,並使在DNN 上的測試準確度平均提高1.01%。 | zh_TW |
dc.description.abstract | Machine learning algorithms, such as Support Vector Machine (SVM) and Deep Neural Network (DNN), have gained a lot of interests recently. When training a machine learning algorithm, randomly shuffle all the training data can improve the testing accuracy and boost the convergence rate. Nevertheless, realizing training data random shuffling in a real system is not a straightforward process due to the slow random accesses in hard disk drive (HDD). To avoid frequent random disk access, the effect of random shuffling is often limited in existing approaches. With the emerging non-volatile memory-based storage device, such as Intel Optane SSD, which provides fast random accesses, we propose a lightweight implementation of random shuffling (LIRS) to randomly shuffle the indexes of the entire training dataset, and the selected training instances are directly accessed from the storage and packed into batches. Experimental results show that LIRS can reduce the total training time of SVM and DNN by 49.9% and 43.5% on average, and improve the final testing accuracy on DNN by 1.01%. | en |
dc.description.provenance | Made available in DSpace on 2021-06-17T06:35:45Z (GMT). No. of bitstreams: 1 ntu-107-R05922083-1.pdf: 1619922 bytes, checksum: 1c5813d2867fe85682faa4b13fd6031e (MD5) Previous issue date: 2018 | en |
dc.description.tableofcontents | 口試委員會審定書 i
誌謝 ii 摘要 iii Abstract iv 1 Introduction 1 2 Background 4 2.1 Basics of Machine Learning Training 4 2.2 Importance of Training Data Random Shuffling 6 3 Motivation 7 3.1 SVM - Block Minimization Framework 7 3.2 DNN - TensorFlow Input Pipeline 8 3.3 Opportunity for Efficient Random Shuffling with NVM-based Storage 9 4 Random Shuffling with SSD 12 4.1 Lightweight Implementation of Random Shuffling (LIRS) 12 4.2 Memory usage analysis 15 4.3 Comparison with conventional approaches 16 5 Evaluation 19 5.1 Experimental Setup 19 5.2 Experimental Results of SVM 21 5.2.1 Convergence Rate and Testing Accuracy 21 5.2.2 Total Training Time 23 5.2.3 Page-aware Random Shuffling vs. Instance-based Random Shuffling 24 5.2.4 Overhead Analysis 25 5.3 Experimental Results of DNN 26 5.3.1 Convergence Rate and Testing Accuracy 26 5.3.2 Total Training Time 27 5.3.3 Overhead Analysis 29 6 Related works 30 7 Conclusion 31 Bibliography 32 | |
dc.language.iso | en | |
dc.title | 在基於非揮發性記憶體的儲存設備上通過輕量級的隨機洗牌方法實現高效率的機器學習 | zh_TW |
dc.title | LIRS: Enabling efficient machine learning on NVM-based storage via a lightweight implementation of random shuffling | en |
dc.type | Thesis | |
dc.date.schoolyear | 106-2 | |
dc.description.degree | 碩士 | |
dc.contributor.oralexamcommittee | 徐慰中(Wei-Chung Hsu),鄭湘筠(Hsiang-Yun Cheng),葉彌妍(Mi-Yen Yeh) | |
dc.subject.keyword | 機器學習,支援向量機,深度神經網路,隨機洗牌,測試準確度,收斂速度,硬式磁碟機,非揮發性記憶體儲存裝置,固態硬碟, | zh_TW |
dc.subject.keyword | Machine learning,Support Vector Machine(SVM),Deep Neural Network(DNN),Random shuffling,Testing accuracy,Convergence rate,Hard disk device(HDD),Non-volatile memory-based storage(NVM-based storage),Solid-state drive(SSD), | en |
dc.relation.page | 35 | |
dc.identifier.doi | 10.6342/NTU201803514 | |
dc.rights.note | 有償授權 | |
dc.date.accepted | 2018-08-16 | |
dc.contributor.author-college | 電機資訊學院 | zh_TW |
dc.contributor.author-dept | 資訊工程學研究所 | zh_TW |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-107-1.pdf 目前未授權公開取用 | 1.58 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。