請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/51244完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 林智仁 | |
| dc.contributor.author | Yong Zhuang | en |
| dc.contributor.author | 庄勇 | zh_TW |
| dc.date.accessioned | 2021-06-15T13:28:23Z | - |
| dc.date.available | 2016-03-08 | |
| dc.date.copyright | 2016-03-08 | |
| dc.date.issued | 2016 | |
| dc.date.submitted | 2016-02-11 | |
| dc.identifier.citation | [1] A. Agarwal, O. Chapelle, M. Dudik, and J. Langford. A reliable effective terascale linear learning system. Journal of Machine Learning Research, 15:1111–1133, 2014.
[2] Y. Bian, X. Li, M. Cao, and Y. Liu. Bundle CDN: a highly parallelized approach for large-scale l1-regularized logistic regression. In Proceedings of European Con- ference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/ PKDD), 2013. [3] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1):1–122, 2011. [4] J. K. Bradley, A. Kyrola, D. Bickson, and C. Guestrin. Parallel coordinate descent for l1-regularized loss minimization. In Proceedings of the Twenty Eighth International Conference on Machine Learning (ICML), pages 321–328, 2011. [5] J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12: 2121–2159, 2011. [6] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIB- LINEAR: a library for large linear classification. Journal of Machine Learn- ing Research, 9:1871–1874, 2008. URL http://www.csie.ntu.edu.tw/~cjlin/ papers/liblinear.pdf. [7] E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V. Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, R. H. Castain, [8] D. J. Daniel, R. L. Graham, and T. S. Woodall. Open MPI: Goals, concept, and design of a next generation MPI implementation. In Proceedings of the 11th European PVM/MPI Users’ Group Meeting, pages 97–104, 2004. [8] S. S. Keerthi and D. DeCoste. A modified finite Newton method for fast solution of large scale linear SVMs. Journal of Machine Learning Research, 6:341–361, 2005. [9] J. Langford, L. Li, and A. Strehl. Vowpal Wabbit, 2007. https://github.com/ JohnLangford/vowpal_wabbit/wiki. [10] C.-J. Lin and J. J. Mor ́e. Newton’s method for large-scale bound constrained problems. SIAM Journal on Optimization, 9:1100–1127, 1999. [11] C.-J. Lin, R. C. Weng, and S. S. Keerthi. Trust region Newton method for large- scale logistic regression. Journal of Machine Learning Research, 9:627–650, 2008. URL http://www.csie.ntu.edu.tw/~cjlin/papers/logistic.pdf. [12] C.-Y. Lin, C.-H. Tsai, C.-P. Lee, and C.-J. Lin. Large-scale logistic regression and linear support vector machines using Spark. In Proceedings of the IEEE International Conference on Big Data, pages 519–528, 2014. URL http://www. csie.ntu.edu.tw/~cjlin/papers/spark-liblinear/spark-liblinear.pdf. [13] D. C. Liu and J. Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(1):503–528, 1989. [14] P. Richtarik and M. Takac. Parallel coordinate descent methods for big data optimization. Mathematical Programming, 2012. Under revision. [15] M. Snir and S. Otto. MPI-the complete reference: the MPI core. MIT Press, Cambridge, MA, USA, 1998. [16] T. Steihaug. The conjugate gradient method and trust regions in large scale optimization. SIAM Journal on Numerical Analysis, 20:626–637, 1983. [17] H.-F. Yu, F.-L. Huang, and C.-J. Lin. Dual coordinate descent methods for logistic regression and maximum entropy models. Machine Learning, 85(1-2):41–75, Octo- ber 2011. URL http://www.csie.ntu.edu.tw/~cjlin/papers/maxent_dual. pdf. [18] G.-X. Yuan, K.-W. Chang, C.-J. Hsieh, and C.-J. Lin. A comparison of optimization methods and software for large-scale l1-regularized linear classification. Journal of Machine Learning Research, 11:3183–3234, 2010. URL http: //www.csie.ntu.edu.tw/~cjlin/papers/l1.pdf. [19] C. Zhang, H. Lee, and K. G. Shin. Efficient distributed linear classification algorithms via the alternating direction method of multipliers. In Proceedings of the 15th International Conference on Artificial Intelligence and Statistics, 2012. [20] Y. Zhuang, W.-S. Chin, Y.-C. Juan, and C.-J. Lin. Distributed Newton method for regularized logistic regression. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), 2015. [21] M. Zinkevich, M. Weimer, A. Smola, and L. Li. Parallelized stochastic gradient descent. In J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. [21] Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 2595–2603. 2010. | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/51244 | - |
| dc.description.abstract | 規則化羅吉斯回歸在分類問題上， 是一個十分有用的方法。 但是對於大規模數據而言， 分散式訓練並沒有被深入的研究。 在這份工作中， 我們提出了分散式牛頓法來訓練羅吉斯回歸。 許多用來降低通訊開銷和加速計算的方法也在本文中進行了討論。 實驗結果表明， 我們提出的和現有最好的方法相比， 是十分有競爭力的。 甚至說是快於現有的方法， 如交替方向乘子法(ADMM)和Vowpal Wabbit (VW)。 我們發佈了一個基於MPI的實現以供大眾使用。 | zh_TW |
| dc.description.abstract | Regularized logistic regression is a very useful classification method, but for large- scale data, its distributed training has not been investigated much. In this work, we propose a distributed Newton method for training logistic regression. Many interesting techniques are discussed for reducing the communication cost and speeding up the computation. Experiments show that the proposed method is competitive with or even faster than state-of-the-art approaches such as Alternating Direction Method of Multipliers (ADMM) and Vowpal Wabbit (VW). We have released an MPI-based implementation for public use. | en |
| dc.description.provenance | Made available in DSpace on 2021-06-15T13:28:23Z (GMT). No. of bitstreams: 1 ntu-105-R01922139-1.pdf: 1357460 bytes, checksum: a05823b24065cfcf926b43b8965c30e5 (MD5) Previous issue date: 2016 | en |
| dc.description.tableofcontents | TABLE OF CONTENTS
口試委員會審定書................................ i 中文摘要...................................... ii ABSTRACT .................................... iii LISTOFFIGURES................................ vi LISTOFTABLES................................. vii CHAPTER I.Introduction ............................... 1 II. Existing Methods for Distributed Classification . . . . . . . . . 3 2.1 ADMMforLogisticRegression .................. 3 2.2 VWforLogisticRegression .................... 5 III.DistributedNewtonMethods..................... 6 3.1 NewtonMethods.......................... 6 3.2 Instance-wise and Feature-wise Data Splits . . . . . . . . . . . 8 3.2.1 Instance-wiseSplit.................... 9 3.2.2 Feature-wiseSplit.................... 9 3.2.3 Analysis ......................... 10 3.3 OtherImplementationTechniques ................ 11 3.3.1 LoadBalancing ..................... 11 3.3.2 DataFormat....................... 11 3.3.3 Speeding Up Hessian-vector Product . . . . . . . . . . 12 IV.Experiments ............................... 13 4.1 TruncatedNewtonMethod .................... 13 4.2 ExperimentalSettings....................... 13 4.3 IW versus FW ........................... 14  4.4 Comparison Between State-of-the-art Methods on Function Values 15 4.5 Comparison Between State-of-the-art Methods on Test Accuracy 17 4.6 Speedup............................... 17 V.Conclusion ................................ 21 APPENDICES................................... 23 BIBLIOGRAPHY................................. 39 | |
| dc.language.iso | en | |
| dc.subject | 羅吉斯回歸 | zh_TW |
| dc.subject | 牛頓法 | zh_TW |
| dc.subject | 分散式系統演算法 | zh_TW |
| dc.subject | 牛頓法 | zh_TW |
| dc.subject | 羅吉斯回歸 | zh_TW |
| dc.subject | 分散式系統演算法 | zh_TW |
| dc.subject | Newton method | en |
| dc.subject | Distributed computing | en |
| dc.subject | Newton method | en |
| dc.subject | Logistic regression | en |
| dc.subject | Logistic regression | en |
| dc.subject | Distributed computing | en |
| dc.title | 分散式牛頓法在規則化羅吉斯回歸上之應用 | zh_TW |
| dc.title | Distributed Newton Method for Regularized Logistic Regression | en |
| dc.type | Thesis | |
| dc.date.schoolyear | 104-1 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 李育杰,林軒田 | |
| dc.subject.keyword | 羅吉斯回歸,牛頓法,分散式系統演算法, | zh_TW |
| dc.subject.keyword | Logistic regression,Newton method,Distributed computing, | en |
| dc.relation.page | 40 | |
| dc.rights.note | 有償授權 | |
| dc.date.accepted | 2016-02-13 | |
| dc.contributor.author-college | 電機資訊學院 | zh_TW |
| dc.contributor.author-dept | 資訊工程學研究所 | zh_TW |
| 顯示於系所單位： | 資訊工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-105-1.pdf 未授權公開取用 | 1.33 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。