請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/72823
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 陳素雲 | |
dc.contributor.author | Toshinari Morimoto | en |
dc.contributor.author | 森元俊成 | zh_TW |
dc.date.accessioned | 2021-06-17T07:07:14Z | - |
dc.date.available | 2019-08-15 | |
dc.date.copyright | 2019-07-25 | |
dc.date.issued | 2019 | |
dc.date.submitted | 2019-07-24 | |
dc.identifier.citation | [1] Chainer: A flexible framework for neural networks. https://chainer.org/. (Accessed on 02/28/2019).
[2] Pytorch. https://pytorch.org/. (Accessed on 07/05/2019). [3] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org. [4] J. C. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121–2159, 07 2011. [5] F. Chollet et al. Keras. https://keras.io, 2015. [6] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization, 2014. cite arxiv:1412.6980Comment: Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015. [7] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, NIPS’12, pages 1097–1105, USA, 2012. Curran Associates Inc. [8] L. De Lathauwer and B. De Moor and J. Vandewalle. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl., 21(4):1253–1278, 2000. [9] Y. LeCun and C. Cortes. MNIST handwritten digit database. 2010. [10] H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos. MPCA: Multilinear Principal Component Analysis of Tensor Objects. Trans. Neur. Netw., 19(1):18–39, Jan. 2008. [11] J. R. Magnus and H. Neudecker. The commutation matrix: Some properties and applications. Tilburg University, Open Access publications from Tilburg University, 7, 03 1979. | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/72823 | - |
dc.description.abstract | 本研究提出了一個卷積神經網路中取代池化的降維方法。池化層是接在卷積層後面,並發揮維度縮減的作用。目前,最大池化或平均池化等的方法被廣泛使用,而我們提出的方法將卷積層的輸出利用截斷的正交矩陣來轉換為維度較小的矩陣。我們將該截斷的正交矩陣視為神經網路中的訓練參數,並推導反向傳播演算法中出現的相關微分。除此以外,我們實際將上述所提的方法寫為電腦程式,驗證其可行性;同時,針對上述所提的方法與池化方法,於盡量相同的條件下進行比較。在實驗中,我們的方法展現較池化方法佳的性能。 | zh_TW |
dc.description.abstract | In this research, we proposed a dimensionality reduction method that takes the place of the pooling methods. A pooling layer is usually put after a convolutional layer to summarize the output images from the convolutional layer. At the moment, the max-pooling method or the average-pooling method is widely used on CNN. On the other hand, our proposed method transforms an output image from a convolutional layer into a lower-dimensional image by multiplying truncated orthogonal matrices. We regard the truncated orthogonal matrices as parameters of the neural network, and we derived the derivatives that appear in the backpropagation algorithm. Moreover, we also verified the feasibility of our proposed method by implementing it as a computer program. We compared the performance of our proposed method with the pooling methods under similar conditions. In the experiment, our proposed method achieved better performance than the pooling methods. | en |
dc.description.provenance | Made available in DSpace on 2021-06-17T07:07:14Z (GMT). No. of bitstreams: 1 ntu-108-R05246013-1.pdf: 541071 bytes, checksum: 532ef952086edacc45d2661465c18bda (MD5) Previous issue date: 2019 | en |
dc.description.tableofcontents | 口試委員會審定書 ii
誌謝 iii Acknowledgements iv 摘要 v Abstract vi 1 Purpose of this Research and Related Works 1 1.1 PCA, MPCA, and High-Order SVD 1 1.2 Pooling Methods vs Our Proposed Method 2 2 Classification based on Neural Network 4 2.1 Basic Idea 4 2.2 Fully Connected Layer 5 2.3 Classification Methods 5 2.3.1 Logistic Regression 5 2.3.2 Neural Network 7 2.4 Convolutional Layer and Pooling Layer 8 2.4.1 Notations 8 2.4.2 Convolutional Layer 8 2.4.3 Pooling Layer 10 2.5 Optimization 11 2.5.1 Gradient Descent 11 2.5.2 Backpropagation Method 12 3 Projective Dimensionality Reduction Layer 15 3.1 Projection Layer (I) Fixed Penalty 15 3.1.1 Forward Path 15 3.1.2 Backward Path 16 3.1.3 Derivation 18 3.2 Projection Layer (II) Learnable Penalty Parameter 23 3.2.1 Forward Path 23 3.2.2 Backward Path 23 3.2.3 Derivation 23 3.3 Additional Statement on Our Methodology 24 4 Implementation 26 4.1 Chainer 26 4.2 Some Mathematical Techniques 28 4.2.1 Vectorization and Restoration 28 4.2.2 Commutation Matrix 29 4.3 Some Comments on Our Code 30 5 Evaluation for the Proposed Method 31 5.1 MNIST handwritten database 31 5.2 Structure of Neural Network and Training Method 31 5.3 Result 33 5.4 Analysis 33 6 Contributions 34 6.1 Contributions 34 6.1.1 List of Contributions 34 6.1.2 Review on Our Work 34 6.2 Future Tasks 35 A Program Source 37 A.1 Projection Layer (I) 37 A.2 Projection Layer (II) 42 B Linear Dimensionality Reduction Layer 49 B.1 Linear Dimensionality Reduction Layer 49 B.1.1 Forward Path 49 B.1.2 Backward Path 50 B.1.3 Test Conditions 50 B.1.4 Test Result 50 B.1.5 Program Code 50 Bibliography 55 | |
dc.language.iso | en | |
dc.title | 卷積神經網路中基於投影的維度縮減層 | zh_TW |
dc.title | A Dimensionality Reduction Layer by Projection in a Convolutional Neural Network | en |
dc.type | Thesis | |
dc.date.schoolyear | 107-2 | |
dc.description.degree | 碩士 | |
dc.contributor.oralexamcommittee | 陳宏,王偉仲,盧鴻興 | |
dc.subject.keyword | 卷積神經網路,維度縮減,池化,截斷正交矩陣,投影,反向傳播演算法, | zh_TW |
dc.subject.keyword | Convolutional Neural Network,Dimensionality Reduction,Pooling,Truncated Orthogonal Matrix,Projection,Backpropagation Algorithm, | en |
dc.relation.page | 56 | |
dc.identifier.doi | 10.6342/NTU201901691 | |
dc.rights.note | 有償授權 | |
dc.date.accepted | 2019-07-24 | |
dc.contributor.author-college | 理學院 | zh_TW |
dc.contributor.author-dept | 應用數學科學研究所 | zh_TW |
顯示於系所單位: | 應用數學科學研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-108-1.pdf 目前未授權公開取用 | 528.39 kB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。