請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/7542
標題: | 牛頓法於卷積神經網路之應用 Newton Methods For Convolutional Neural Networks |
作者: | Kent Loong Tan 陳勁龍 |
指導教授: | 林智仁(Chih-Jen Lin) |
關鍵字: | 卷積神經網路,多類別分類,大規模學習,抽樣海森矩陣, Convolutional neural networks,multi-class classification,large-scale classification,subsampled Hessian, |
出版年 : | 2018 |
學位: | 碩士 |
摘要: | 深度學習包含困難的非凸優化問題。大多數研究經常使用隨機梯度演算法(SG)來優化這類模型。使用SG通常很有效,但有時並不那麼強大。近代的研究探討了利用牛頓法作為替代的優化方法,但絕大部分研究只將其應用於全連接神經網路。他們沒有探討諸如卷積神經網路等更為廣泛使用的深度學習模型。其中一個原因是應用牛頓法於卷積神經網路的過程中牽涉到多個複雜的運算,因此目前未有仔細的相關研究。在這篇論文中,我們給出詳細的建構模組,當中包括函數、梯度及賈可比矩陣的運算和高斯-牛頓矩陣向量的乘積。這些基本的模組非常重要。因為沒有它們,任何牛頓法於全連接神經網路的進步沒辦法在卷積神經網路上嘗試。因此我們的研究將可能推動更多牛頓法於卷積神經網路上的發展。我們完成一個簡單的MATLAB實作。實驗結果顯示這個方法具有競爭力。 Deep learning involves a difficult non-convex optimization problem, which is often solved by stochastic gradient (SG) methods. While SG is usually effective, it is sometimes not very robust. Recently, Newton methods have been investigated as an alternative optimization technique, but nearly all existing studies consider only fully-connected feedforward neural networks. They do not investigate other types of networks such as Convolutional Neural Networks (CNN), which are more commonly used in deep-learning applications. One reason is that Newton methods for CNN involve complicated operations, and so far no works have conducted a thorough investigation. In this thesis, we give details of building blocks including function, gradient, and Jacobian evaluation, and Gauss-Newton matrix-vector products. These basic components are very important because without them none of any recent improvement of Newton methods for fully-connected networks can even be tried. Thus we will enable possible further developments of Newton methods for CNN. We finish a simple MATLAB implementation and show that it gives competitive test accuracy. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/7542 |
DOI: | 10.6342/NTU201802108 |
全文授權: | 同意授權(全球公開) |
顯示於系所單位: | 資訊網路與多媒體研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-107-1.pdf | 1.35 MB | Adobe PDF | 檢視/開啟 |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。