請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/99508完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 簡韶逸 | zh_TW |
| dc.contributor.advisor | Shao-Yi Chien | en |
| dc.contributor.author | 杜冠廷 | zh_TW |
| dc.contributor.author | Kuan-Ting Tu | en |
| dc.date.accessioned | 2025-09-10T16:30:25Z | - |
| dc.date.available | 2025-09-11 | - |
| dc.date.copyright | 2025-09-10 | - |
| dc.date.issued | 2025 | - |
| dc.date.submitted | 2025-07-08 | - |
| dc.identifier.citation | [1] J. Zamora, J. A. Cruz Vargas, A. Rhodes, L. Nachman, and N. Sundararajan, “Convolutional filter approximation using fractional calculus,” in Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021, pp. 383–392. ix, 3, 8, 9, 19, 24, 28
[2] A.-H. Phan, K. Sobolev, K. Sozykin, D. Ermilov, J. Gusak, P. Tichavsk `y, V. Glukhov, I. Oseledets, and A. Cichocki, “Stable low-rank tensor decomposition for compression of convolutional neural network,” in Proceedings of the European Conference on Computer Vision (ECCV). Springer, 2020, pp. 522–539 [3] Y. Sui, M. Yin, Y. Gong, J. Xiao, H. Phan, and B. Yuan, “Elrt: Efficient low-rank training for compact convolutional neural networks,” arXiv preprint arXiv:2401.10341, 2024. ix, 11, 12, 41, 42 [4] K. Guo, Z. Lin, C. Chen, X. Xing, F. Liu, and X. Xu, “Compact model training by low-rank projection with energy transfer,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, 2024. ix, 12, 13, 43, 44 [5] C. Tai, T. Xiao, X. Wang, and W. E, “Convolutional neural networks with low-rank regularization,” in International Conference on Learning Representations (ICLR), 2016. ix, 11, 13, 14 [6] X. Ruan, Y. Liu, C. Yuan, B. Li, W. Hu, Y. Li, and S. Maybank, “Edp: An efficient decomposition and pruning scheme for convolutional neural network compression,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 10, pp. 4499–4513, 2024. ix, 14, 16, 44 [7] V. T. Pham, Y. Zniyed, and T. P. Nguyen, “Enhanced network compression through tensor decompositions and pruning,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–13, 2024. ix, 14, 17, 44 [8] Y. Li, S. Lin, J. Liu, Q. Ye, M. Wang, F. Chao, F. Yang, J. Ma, Q. Tian, and R. Ji, “Towards compact cnns via collaborative compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 6434–6443. ix, 15, 17, 44 [9] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems, 2012. 1 [10] R. Gonzalez and R. Woods, Digital Image Processing. Prentice Hall, 2008. 3, 23 [11] N. Kanopoulos, N. Vasanthavada, and R. Baker, “Design of an image edge detection filter using the sobel operator,” IEEE Journal of Solid-State Circuits, vol. 23, no. 2, pp. 358–367, 1988. 3, 23 [12] H. Jalalinejad, A. Tavakoli, and F. Zarmehi, “A simple and flexible modification of grünwald–letnikov fractional derivative in image processing,” Mathematical Sciences, vol. 12, no. 3, pp. 205–210, 2018. 3, 7, 23 [13] H. Jinrong, P. Yifei, and Z. Jiliu, “Construction of fractional differential masks based on riemann–liouville definition,” Journal of Computational Information Systems, vol. 6, no. 10, pp. 3191–3199, 2010. 7 [14] ——, “A novel image denoising algorithm based on riemann-liouville definition,” Journal of Computers, vol. 6, no. 7, pp. 1332–1338, 2011. 7 [15] H. A. Jalab and R. W. Ibrahim, “Fractional masks based on generalized fractional differential operator for image denoising,” International Journal of Computer and Information Engineering, vol. 7, no. 2, pp. 308–313, 2013. 7 [16] S. Mishra, K. Singh, R. Dixit, and M. Bajpai, “Design of fractional calculus based differentiator for edge detection in color images,” Multimedia Tools and Applications, vol. 80, pp. 29 965–29 983, 2021. 7 [17] T. Pragnesh and B. R. Mohan, “Compression of convolution neural network using structured pruning,” in Proceedings of the IEEE 7th International conference for Convergence in Technology (I2CT), 2022, pp. 1–5. 9 [18] Y. Song, B. Wu, T. Yuan, and W. Liu, “A high-speed cnn hardware accelerator with regular pruning,” in Proceedings of the 23rd International Symposium on Quality Electronic Design (ISQED), 2022, pp. 1–5. 10 [19] Z. Liu, J. Xu, X. Peng, and R. Xiong, “Frequency-domain dynamic pruning for convolutional neural networks,” in Advances in Neural Information Processing Systems, 2018. 10 [20] S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” in Advances in Neural Information Processing Systems, 2015. 10 [21] J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks,” in International Conference on Learning Representations (ICLR), 2019. 10 [22] T. Zhang, S. Ye, K. Zhang, J. Tang, W. Wen, M. Fardad, and Y. Wang, “A systematic dnn weight pruning framework using alternating direction method of multipliers,” in Proceedings of the European Conference on Computer Vision (ECCV). Springer, 2018, pp. 184–199. 10 [23] M. Lin, R. Ji, Y. Wang, Y. Zhang, B. Zhang, Y. Tian, and L. Shao, “Hrank: Filter pruning using high-rank feature map,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 1526–1535. 11, 44 [24] Y. Sui, M. Yin, Y. Xie, H. Phan, S. Aliari Zonouz, and B. Yuan, “Chip: Channel independence-based pruning for compact neural networks,” in Advances in Neural Information Processing Systems, 2021, pp. 24 604–24 616. 11, 44 [25] L. Liebenwein, C. Baykal, H. Lang, D. Feldman, and D. Rus, “Provable filter pruning for efficient neural networks.” in International Conference on Learning Representations (ICLR), 2020. 11, 43 [26] Y. Tang, Y. Wang, Y. Xu, D. Tao, C. XU, C. Xu, and C. Xu, “Scop: Scientific control for reliable neural network pruning,” in Advances in Neural Information Processing Systems, 2020, pp. 10 936–10 947. 11, 41, 42, 43 [27] M. Lin, R. Ji, Y. Zhang, B. Zhang, Y. Wu, and Y. Tian, “Channel pruning via automatic structure search,” in Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI), 2021, pp. 673–679. 11, 44 [28] H. Wang, P. Ling, X. Fan, T. Tu, J. Zheng, H. Chen, Y. Jin, and E. Chen, “All-in-one hardware-oriented model compression for efficient multi-hardware deployment,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 12, pp. 12 345–12 359, 2024. 11, 44 [29] T. Yuan, Z. Li, B. Liu, Y. Tang, and Y. Liu, “Arpruning: An automatic channel pruning based on attention map ranking,” Neural Networks, vol. 174, p. 106220, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0893608024001448 11, 44 [30] V. T. Pham, Y. Zniyed, and T. P. Nguyen, “Efficient tensor decomposition-based filter pruning,” Neural Networks, vol. 178, p. 106393, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0893608024003174 11, 44 [31] L. Yang, S. Gu, C. Shen, X. Zhao, and Q. Hu, “Soft independence guided filter pruning,” Pattern Recognition, vol. 153, p. 110488, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320324002395 11, 44 [32] X. Yuan, P. Savarese, and M. Maire, “Growing efficient deep networks by structured continuous sparsification,” in International Conference on Learning Representations (ICLR), 2021. 11, 41, 42, 44 [33] X. Zhou, W. Zhang, Z. Chen, S. DIAO, and T. Zhang, “Efficient neural network training via forward and backward propagation sparsification,” in Advances in Neural Information Processing Systems, 2021, pp. 15 216–15 229. 11, 41, 42, 44 [34] E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” in Advances in Neural Information Processing Systems, 2014. 11 [35] B.-S. Chu and C.-R. Lee, “Low-rank tensor decomposition for compression of convolutional neural networks using funnel regularization,” arXiv preprint arXiv:2112.03690, 2021. 11, 43 [36] N. Li, Y. Pan, Y. Chen, Z. Ding, D. Zhao, and Z. Xu, “Heuristic rank selection with progressively searching tensor ring network,” Complex & Intelligent Systems, vol. 8, no. 2, pp. 771–785, 2022. 11, 41, 42 [37] W. Liu, P. Liu, C. Shi, Z. Zhang, Z. Li, and C. Liu, “Tdlc: Tensor decomposition-based direct learning-compression algorithm for dnn model compression,” Concurrency and Computation: Practice and Experience, 2024. 12, 41 [38] Y. Li, S. Gu, C. Mayer, L. Van Gool, and R. Timofte, “Group sparsity: The hinge between filter pruning and decomposition for network compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 8015–8024. 14, 41 [39] A. Llanza, F. E. Keddous, N. Shvai, and A. Nakib, “Deep learning models compression based on evolutionary algorithms and digital fractional differentiation,” in Proceedings of the IEEE Congress on Evolutionary Computation (CEC), 2023, pp. 1–9. 19 [40] I. Podlubny, Fractional differential equations: an introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications. elsevier, 1998, vol. 198. 22 [41] R. Scherer, S. L. Kalla, Y. Tang, and J. Huang, “The grünwald–letnikov method for fractional differential equations,” Computers & Mathematics with Applications, vol. 62, no. 3, pp. 902–917, 2011. 23 [42] M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. National Bureau of Standards Applied Mathematics Series 55. Tenth Printing. ERIC, 1972. 23 [43] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. 39, 46, 50, 51 [44] S. Zagoruyko and N. Komodakis, “Wide residual networks,” in Proceedings of the British Machine Vision Conference (BMVC), 2016, pp. 87.1–87.12. 39, 48 | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/99508 | - |
| dc.description.abstract | 隨著深度神經網路在實際應用中的廣泛使用，神經網路壓縮技術的重要性日益提升，特別是在資源受限的邊緣設備上。因為在現實世界的應用中，深度神經網路（DNN）往往對其造成沉重的硬體負擔。
儘管已有諸多方法致力於壓縮模型參數，部署這些模型在邊緣設備上仍面臨諸多挑戰。為了應對這個問題，我們提出了結合分數階高斯濾波器和剪枝的壓縮架構FGFP（Fractional Gaussian Filter and Pruning）。該框架整合了分數階微分和高斯函數以構建分數階高斯濾波器（FGFs）。為了降低分數階微分運算的計算複雜度，我們引入了Grünwald–Letnikov分數階導數來近似分數階微分方程式。每個FGF捲積核的參數量被精簡至僅剩七個。進一步地，為了優化運算效率，我們透過運算順序重排，在分數高斯濾波器架構中實現了卷積運算計算量減少64.06%。除分數高斯濾波器架構外，我們的FGFP架構還整合了自適應非結構化剪枝（AUP）以實現更高的壓縮率。 在各種架構和基準數據集上的實驗表明，我們的FGFP架構在準確性和壓縮率方面都優於最近的方法。在CIFAR-10上，ResNet-20在模型尺寸縮小85.2%的情況下，準確度僅下降了1.52%。在ILSVRC2012上，ResNet-50在模型尺寸縮小69.1%的情況下，準確度僅下降了1.63%。 為了評估該框架的實際適用性，我們在消費級筆記型電腦上部署了該模型進行推理。實驗結果顯示，該方法在端到端推理過程中實現了3.33倍的速度提升。 | zh_TW |
| dc.description.abstract | Neural network compression techniques are becoming increasingly important nowadays because Deep Neural Networks (DNNs) impose heavy hardware resource loads on edge devices in real-world applications.
While many methods compress neural network parameters, deploying these models on edge devices remains challenging. To address this, we propose the Fractional Gaussian Filter and Pruning (FGFP) framework, which integrates fractional-order differential calculus and Gaussian function to construct fractional Gaussian filters (FGFs). To reduce the computational complexity of fractional-order differential operations, we introduce Grünwald-Letnikov fractional derivatives to approximate the fractional-order differential equation. The number of parameters for each kernel in FGF is minimized to only seven. Furthermore, through operation reordering, we achieve a 64.06% reduction in the computational load of convolution operations within the fractional Gaussian filter architecture. Beyond the architecture of fractional Gaussian filters, our FGFP framework also incorporates Adaptive Unstructured Pruning (AUP) to achieve higher compression ratios. Experiments on various architectures and benchmarks show that our FGFP framework outperforms recent methods in accuracy and compression. On CIFAR-10, ResNet-20 achieves only a 1.52% drop in accuracy while reducing the model size by 85.2%. On ILSVRC2012, ResNet-50 achieves only a 1.63% drop in accuracy while reducing the model size by 69.1%. To assess the framework’s real-world applicability, we deployed the model on a consumer laptop, achieving a 3.33× end-to-end speedup. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2025-09-10T16:30:25Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2025-09-10T16:30:25Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | Master's Thesis Acceptance Certificate i
Acknowledgement iii Chinese Abstract v Abstract vii Contents ix List of Figures xi List of Tables xiii 1 Introduction 1 1.1 Deep Neural Network Compression 1 1.2 Challenges 3 1.3 Contribution 4 1.4 Thesis Organization 5 2 Related Work 7 2.1 Fractional Approximation Filter 7 2.2 Neural Network Compression 9 2.2.1 Network Sparsity 9 2.2.2 Low-Rank Representation 11 2.2.3 Hybrid Approach 14 3 Proposed Method 19 3.1 Overall Framework of FGFP 19 3.2 Grünwald-Letnikov Fractional Derivatives 21 3.3 Fractional Gaussian Filter (FGF) 23 3.3.1 Channel-Attention Fractional Gaussian Filter 27 3.3.2 Three-Dimensional Fractional Gaussian Filter 30 3.4 Adaptive Unstructured Pruning(AUP) 32 3.5 Speedup for Inference 35 4 Experiments 39 4.1 Experimental Settings 39 4.2 Performance Evaluation 40 4.2.1 Comparison on CIFAR-10 40 4.2.2 Comparison on ILSVRC2012 43 4.2.3 Inference Speedup Analysis 45 4.3 Ablation Study 50 4.3.1 Comparison of FGFP, AUP, and CA-FGF 50 4.3.2 Comparison of FGFP (CA-FGF) and FGFP (3D-FGF) 50 5 Conclusion 53 5.1 Conclusion 53 5.2 Future Work 54 Reference 55 | - |
| dc.language.iso | en | - |
| dc.subject | 網路壓縮 | zh_TW |
| dc.subject | 分數階導數 | zh_TW |
| dc.subject | Grünwald-Letnikov 分數階導數 | zh_TW |
| dc.subject | 高斯函數 | zh_TW |
| dc.subject | 自適應非結構化剪枝 | zh_TW |
| dc.subject | Adaptive Unstructured Pruning | en |
| dc.subject | Networks Compression | en |
| dc.subject | Fractional-Order Derivative | en |
| dc.subject | Grünwald-Letnikov fractional derivatives | en |
| dc.subject | Gaussian Function | en |
| dc.title | 結合分數階高斯濾波器與剪枝之深度神經網路壓縮方法 | zh_TW |
| dc.title | FGFP: A Fractional Gaussian Filter and Pruning for Deep Neural Networks Compression | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 113-2 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 曹昱;劉宗德;鄭文皇 | zh_TW |
| dc.contributor.oralexamcommittee | Yu Tsao;Tsung-Te Liu;Wen-Huang Cheng | en |
| dc.subject.keyword | 網路壓縮,分數階導數,Grünwald-Letnikov 分數階導數,高斯函數,自適應非結構化剪枝, | zh_TW |
| dc.subject.keyword | Networks Compression,Fractional-Order Derivative,Grünwald-Letnikov fractional derivatives,Gaussian Function,Adaptive Unstructured Pruning, | en |
| dc.relation.page | 60 | - |
| dc.identifier.doi | 10.6342/NTU202501353 | - |
| dc.rights.note | 同意授權(全球公開) | - |
| dc.date.accepted | 2025-07-10 | - |
| dc.contributor.author-college | 重點科技研究學院 | - |
| dc.contributor.author-dept | 積體電路設計與自動化學位學程 | - |
| dc.date.embargo-lift | 2030-06-26 | - |
| 顯示於系所單位： | 積體電路設計與自動化學位學程 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-113-2.pdf 此日期後於網路公開 2030-06-26 | 3.81 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。