請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/74657
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 韓仁毓 | |
dc.contributor.author | Yen-Ju Huang | en |
dc.contributor.author | 黃彥儒 | zh_TW |
dc.date.accessioned | 2021-06-17T08:48:17Z | - |
dc.date.available | 2024-08-07 | |
dc.date.copyright | 2019-08-07 | |
dc.date.issued | 2019 | |
dc.date.submitted | 2019-08-05 | |
dc.identifier.citation | Cortes, C. & Vapnik, V. (1995). Support Vector Networks. Machine Learning, 20, 273-297.
Dechter, R. (1986). Learning While Searching in Constraint- atisfaction- roblems. AAAI Dinh, L., Pascanu, R., Bengio, S. & Bengio, Y. (2017). Sharp Minima Can Generalize For Deep Nets.. CoRR, abs/1703.04933. Dauphin, Y. N., Pascanu, R., Gülçehre, Ç., Cho, K., Ganguli, S. & Bengio, Y. (2014). Identifying and attacking the saddle point problem in high-dimensional non-convex optimization.. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence & K. Q. Weinberger (eds.), NIPS (p./pp. 2933-2941), . Duchi, J. C., Hazan, E. & Singer, Y. (2011). Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.. Journal of Machine Learning Research, 12, 2121-2159. Dumoulin, V. & Visin, F. (2018). A guide to convolution arithmetic for deep learning (cite arXiv:1603.07285v2) Erhan, D., Bengio, Y., Courville, A. & Vincent, P. (2009). Visualizing higher-layer features of a deep network. , . Glorot, X. & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks.. In Y. W. Teh & D. M. Titterington (eds.), AISTATS (p./pp. 249-256), : JMLR.org. Goodfellow, I. J. & Vinyals, O. (2015). Qualitatively characterizing neural network optimization problems.. In Y. Bengio & Y. LeCun (eds.), ICLR, . Ganin, Y. & Lempitsky, V. S. (2015). Unsupervised Domain Adaptation by Backpropagation.. In F. R. Bach & D. M. Blei (eds.),ICML (p./pp. 1180-1189), : JMLR.org. Huang, G., Liu, Z. & Weinberger, K. Q. (2016). Densely Connected Convolutional Networks.. CoRR, abs/1608.06993. Hoffer, E., Hubara, I. & Soudry, D. (2017). Train longer, generalize better: closing the generalization gap in large batch training of neural networks.. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan & R. Garnett (eds.),NIPS (p./pp. 1729-1739), . Hinton, G., Srivastava, N., & Swersky, K. (2016). Neural Networks for Machine Learning Lecture 6a Overview of mini-batch gradient descent. Retrieved from https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf He, K., Zhang, X., Ren, S. & Sun, J. (2015). Deep Residual Learning for Image Recognition (cite arxiv:1512.03385Comment: Tech report) Ioffe, S. & Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.. CoRR, abs/1502.03167. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016. Jastrzebski, S., Kenton, Z., Arpit, D., Ballas, N., Fischer, A., Bengio, Y. & Storkey, A. J. (2017). Three Factors Influencing Minima in SGD.. CoRR, abs/1711.04623. Kirillov, A., He, K., Girshick, R., Rother, C., & Dollar, P. (2019). Panoptic Segmentation. Retrieved from https://arxiv.org/pdf/1801.00868.pdf Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems (p./pp. 1097--1105), . Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems (p./pp. 1097--1105), . Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M. & Tang, P. T. P. (2016). On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima (cite arxiv:1609.04836) Kingma, D. P. & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, . Krogh, A. & Hertz, J. A. (1992). A simple weight decay can improve generalization. In D. S. Lippman, J. E. Moody & D. S. Touretzky (eds.), Advances in Neural Information Processing Systems 4 (p./pp. 950--957), : Morgan Kaufmann. Lee, H. (2016). Backpropagation. Retrieved from http://speech.ee.ntu.edu.tw/~tlkagk /courses/ML_2016/ Lecture/BP.pdf Li, F., Johnson, J., & Yeung, S. (2017). Lecture 7: Training Neural Networks, Part 2. Retrieved from http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture7.pdf Li, F., Johnson, J., & Yeung, S. (2017). Lecture 12: Visualizing and Understanding. Retrieved from http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture12.pdf Li, F., Johnson, J., & Yeung, S. (2017). Lecture 15: Convolutional Neural Networks. Retrieved from http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture5.pdf Li, F., Johnson, J., & Yeung, S. (2017). Lecture 11: Detection and Segmentation. Retrieved from http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture11.pdf Loshchilov, I., & Hutter, F. (2019). Decoupled Weight Decay Regularization. Retrieved from https://arxiv.org/abs/1711.05101 Mascagni, M., Aart, E., & Korst, J. (1990). Simulated Annealing and Boltzmann Machines: A Stochastic Approach to Combinatorial Optimization and Neural Computing. Mathematics Of Computation, 55(191), 393. doi: 10.2307/2008816 Neelakantan, A., Vilnis, L., Le, Q. V., Sutskever, I., Kaiser, L., Kurach, K. & Martens, J. (2015). Adding Gradient Noise Improves Learning for Very Deep Networks.. CoRR, abs/1511.06807. Qian, N. (1999). On the momentum term in gradient descent learning algorithms.. Neural Networks, 12, 145-151. Robbins, H., & Monro, S. (1951). A Stochastic Approximation Method. The Annals Of Mathematical Statistics, 22(3), 400-407. doi: 10.1214/aoms/1177729586 Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-536. doi: 10.1038/323533a0 Ronneberger, O., Fischer, P. & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation.. CoRR, abs/1505.04597. Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85-117. doi: 10.1016/j.neunet.2014.09.003 Smith, L. N. & Topin, N. (2017). Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates (cite arxiv:1708.07120Comment: This paper was significantly revised to show super-convergence as a general fast training methodology) Simonyan, K. & Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR, abs/1409.1556. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. & Rabinovich, A. (2015).Going deeper with convolution s. Proceedings of the IEEE conference on computer vision and pattern recognition (p./pp. 1--9), . Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D. & Wang, Z. (2016). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network.. CVPR (p./pp. 1874-1883), : IEEE Computer Society. ISBN: 978-1-4673-8851 Smith, L. N. (2017). Cyclical Learning Rates for Training Neural Network s.. WAC V (p./pp. 464-472), : IEEE Computer Society. ISBN: 978-1-5090-4822-9 Smith, L. (2018). A DISCIPLINED APPROACH TO NEURAL NETWORK HYPER -PARAMETERS: PART 1 – LEARNING RATE, BATCH SIZE, MOMENTUM, AND WEIGHT DECAY. Retrieved from https://arxiv.org/pdf/1803.09820.pdf Sutskever, I., Martens, J., Dahl, G. E. & Hinton, G. E. (2013). On the importance of initialization and momentum in deep learning.. ICML (3) (p./pp. 1139-1147), : JMLR.org. Simonyan, K., Vedaldi, A. & Zisserman, A. (2013). Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps.. CoRR, abs/1312.6034. Santurkar, S., Tsipras, D., Ilyas, A. & Madry, A. (2018). How Does Batch Normalization Help Optimization?. In S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi & R. Garnett (eds.), NeurIPS (p./pp. 2488-2498), . Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D. & Wang, Z. (2016). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network.. CVPR (p./pp. 1874-1883), : IEEE Computer Society. ISBN: 978-1-4673-8851-1 van der Maaten, L. & Hinton, G. (2008). Visualizing Data using t-SNE . Journal of Machine Learning Research, 9, 2579--2605. Yosinski, J., Clune, J., Bengio, Y. & Lipson, H. (2014). How transferable are features in deep neural networks?. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence & K. Q. Weinberger (eds.), NIPS (p./pp. 3320-3328), . Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41–48. ACM, 2009. 林軒田. (2015). 機器學習基石 Lecture 14: Regularization. Retrieved from https://www.csie.ntu.edu.tw/~htlin/mooc/doc/14_present.pdf | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/74657 | - |
dc.description.abstract | 在工程問題中,常常會利用影像分割的技術來幫助解決真實世界之工程問題,透過對於河床影像之分割來輔助獲取河道空間及屬性進而作為橋梁安全的評估即為本研究之重要工作之一。在傳統的影像處理技術中,要完成這種多群之分割,往往需要人工調整不同的分割演算法的參數;近年來,雖然深度學習的技術有了重大突破,於影像分割之應用也屢見不鮮,但是其模型的訓練上往往需要大量的時間、資料。本研究透過近年來深度學習幾個於電腦視覺領域中具突破性之訓練策略、模型設計以及超參數和超參數間之搭配來優化訓練演算法。此外,為了測試以及檢驗演算法之可用性以及性能,本研究利用2018年CVPR所辦比賽提供的空拍影像資料來做為本研究之比較、輔助驗證。透過本研究採用之訓練演算法,相較於深度學習傳統方式的訓練,不僅大幅減少至少三倍所需之訓練時間,其準確度更提高1~2%的精度。 | zh_TW |
dc.description.abstract | In the real world problem, performing image semantic segmentation can be helpful to solve real world problem, for example, this technology could potentially apply on gaining information of riverbed as one of indicator for monitoring bridge state. In traditional image processing algorithm, we always need to tune the parameters in the algorithm manually. Deep learning technology has big breakthrough over the past few years, it’s often to see people apply it on image segmentation. However, it always takes lots of time and data for training the model. This research takes the combination of break-through training strategies of deep learning in computer vision field and seeking to decrease the time spending for training our model. Furthermore, we examine the training algorithm by two sets of data including ours and from the competition hold by CVPR in 2018. We successfully make our model converge three times faster than what it used to take and even outperform the model trained with traditional method by 1~2% of accuracy. | en |
dc.description.provenance | Made available in DSpace on 2021-06-17T08:48:17Z (GMT). No. of bitstreams: 1 ntu-108-R05543071-1.pdf: 6321851 bytes, checksum: 43756b687934e7ed7bfe7605e8f78fc3 (MD5) Previous issue date: 2019 | en |
dc.description.tableofcontents | 誌謝 i
中文摘要 ii ABSTRACT iii 目錄 iv 圖目錄 vii 表目錄 x 第一章 緒論 1 1.1 研究背景 1 1.2 研究動機與目的 2 1.3 論文架構 3 第二章 文獻回顧 4 2.1影像分割於深度學習種類 4 2.2深度學習參數更新演算法 5 2.2.1神經網路 5 2.2.2損失函數(Loss function) 6 2.2.3參數更新(Parameters Update) 6 2.3超參數(Hyper-Parameters) 9 2.3.1週期性學習率 9 2.3.2權重衰減(Weight Decay) 11 2.4 優化器(Optimizer) 13 2.4.1隨機梯度下降優化器(Stochastic Gradient Descent) 13 2.4.2 AdaGrad 14 2.4.3 RMSProp 15 2.4.4 SGD + Momentum 16 2.4.5 Adam 17 2.5 One Cycle Policy 18 2.6神經網路 – UNET 20 2.6.1 Encoder 22 2.6.2 Decoder 26 2.6.3 Batch Normalization 29 2.7遷移學習(Transfer Learning) 33 罩窗可視化( Visualize The Kernels ) 34 梯度上升(Gradient Ascent)生成影像 34 2.8 文獻回顧總結 39 第三章 研究方法 40 3.1 One Cycle Policy模型訓練演算法 42 3.1.1輸入影像 42 3.1.2神經網路 42 3.1.3學習率&動量之組合 44 3.2訓練演算法優化 46 3.2.1 學習率之變化率以及初始值影響探討 46 3.2.2演算法受訓練長久影響 53 3.3河床分佈影像之分割 53 3.3.1 資料增強(Data augmentation) 53 3.3.2 VGG16 54 3.3.3 One Cycle Policy 55 第四章 實驗及成果分析 56 4.1 One Cycle Policy與傳統訓練之比較 56 4.1.1 傳統訓練之模型表現 56 4.1.2 One Cycle Policy訓練之模型表現 57 4.2參數對於One Cycle Policy之影響 58 4.2.1固定住變化率因子變動初始因子 58 4.2.2固定住初始因子變動變化率因子 59 4.2.3訓練加長,模型出現不穩定現象 59 4.2.4調整超參數更有效全面性提升準確度 63 4.3應用於河床分佈影像成果 74 4.3.1河床影像作為訓練資料 74 4.3.2傳統訓練之模型表現 75 4.3.3 One Cycle Policy訓練之模型表現 76 4.3.4河床分割結果 77 4.3.5資料域特性影響模型表現 78 第五章 結論與建議 80 5.1 結論與建議 80 5.2 未來工作 81 REFERENCE 82 | |
dc.language.iso | zh-TW | |
dc.title | 以快速收斂 UNET 深度學習之模型進行河床之影像分割 | zh_TW |
dc.title | Performing Semantic Segmentation On Riverbed With FastConvergent UNET Deep Neural Network. | en |
dc.type | Thesis | |
dc.date.schoolyear | 107-2 | |
dc.description.degree | 碩士 | |
dc.contributor.oralexamcommittee | 陳俊杉,何昊哲 | |
dc.subject.keyword | 深度學習,影像分割,快速收斂, | zh_TW |
dc.subject.keyword | deep learning,semantic segmentation,fast convergence, | en |
dc.relation.page | 85 | |
dc.identifier.doi | 10.6342/NTU201902238 | |
dc.rights.note | 有償授權 | |
dc.date.accepted | 2019-08-05 | |
dc.contributor.author-college | 工學院 | zh_TW |
dc.contributor.author-dept | 土木工程學研究所 | zh_TW |
顯示於系所單位: | 土木工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-108-1.pdf 目前未授權公開取用 | 6.17 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。