請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/87975完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 陳尚澤 | zh_TW |
| dc.contributor.advisor | Shang-Tse Chen | en |
| dc.contributor.author | 吳由由 | zh_TW |
| dc.contributor.author | Yu-Yu Wu | en |
| dc.date.accessioned | 2023-08-01T16:10:48Z | - |
| dc.date.available | 2023-11-09 | - |
| dc.date.copyright | 2023-08-01 | - |
| dc.date.issued | 2023 | - |
| dc.date.submitted | 2023-06-28 | - |
| dc.identifier.citation | [1] S. Addepalli, S. Jain, and R. V. Babu. Efficient and effective augmentation strategy for adversarial training. CoRR, abs/2210.15318, 2022.
[2] M. Andriushchenko, F. Croce, N. Flammarion, and M. Hein. Square attack: A query efficient black-box adversarial attack via random search. In A. Vedaldi, H. Bischof, T. Brox, and J. Frahm, editors, Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII, volume 12368 of Lecture Notes in Computer Science, pages 484–501. Springer, 2020. [3] A. Athalye, N. Carlini, and D. A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In J. G. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 274–283. PMLR, 2018. [4] N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber, D. Tsipras, I. Goodfellow, A. Madry, and A. Kurakin. On evaluating adversarial robustness. arXiv:1902.06705, 2019. [5] Y. Carmon, A. Raghunathan, L. Schmidt, J. C. Duchi, and P. Liang. Unlabeled data improves adversarial robustness. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. B. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 11190–11201, 2019. [6] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin. Emerging properties in self-supervised vision transformers. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 9630–9640. IEEE, 2021. [7] E. Chen and C. Lee. LTD: low temperature distillation for robust adversarial training. CoRR, abs/2111.02331, 2021. [8] T. Chen, Z. Zhang, S. Liu, S. Chang, and Z. Wang. Robust overfitting may be mitigated by properly learned smoothening. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. [9] J. M. Cohen, E. Rosenfeld, and J. Z. Kolter. Certified adversarial robustness via randomized smoothing. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 1310–1320. PMLR, 2019. [10] F. Croce and M. Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 2196–2205. PMLR, 2020. [11] F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an en- semble of diverse parameter-free attacks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, vol- ume 119 of Proceedings of Machine Learning Research, pages 2206–2216. PMLR, 2020. [12] F. Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chi- ang, P. Mittal, and M. Hein. Robustbench: a standardized adversarial robustness benchmark. In J. Vanschoren and S. Yeung, editors, Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021. [13] J. Cui, S. Liu, L. Wang, and J. Jia. Learnable boundary guided adversarial train- ing. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 15701–15710. IEEE, 2021. [14] J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, pages 248–255. IEEE Computer Society, 2009. [15] C. Dong, L. Liu, and J. Shang. Label noise in adversarial training: A novel per- spective to study robust overfitting. In Advances in Neural Information Processing Systems, 2022. [16] Y. Dong, K. Xu, X. Yang, T. Pang, Z. Deng, H. Su, and J. Zhu. Exploring memo- rization in adversarial training. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. [17] L. Engstrom, A. Ilyas, and A. Athalye. Evaluating and understanding the robustness of adversarial logit pairing. CoRR, abs/1807.10272, 2018. [18] T. Garipov, P. Izmailov, D. Podoprikhin, D. P. Vetrov, and A. G. Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. In S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 8803–8812, 2018. [19] M. Goldblum, L. Fowl, S. Feizi, and T. Goldstein. Adversarially robust distilla- tion. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 3996– 4003. AAAI Press, 2020. [20] S. Gowal, C. Qin, J. Uesato, T. A. Mann, and P. Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. CoRR, abs/ 2010.03593, 2020. [21] J. Grabinski, P. Gavrikov, J. Keuper, and M. Keuper. Robust models are less over- confident. CoRR, abs/2210.05938, 2022. [22] J. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Do- ersch, B. Á. Pires, Z. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko. Bootstrap your own latent - A new approach to self-supervised learn- ing. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. [23] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770–778. IEEE Computer Society, 2016. [24] K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In B. Leibe, J. Matas, N. Sebe, and M. Welling, editors, Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV, volume 9908 of Lecture Notes in Computer Science, pages 630–645. Springer, 2016. [25] M. Hein and M. Andriushchenko. Formal guarantees on the robustness of a clas- sifier against adversarial manipulation. In I. Guyon, U. von Luxburg, S. Ben- gio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 2266–2276, 2017. [26] D. Hendrycks and K. Gimpel. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. CoRR, abs/1606.08415, 2016. [27] G. E. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. CoRR, abs/1503.02531, 2015. [28] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. InH. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. [29] P. Izmailov, D. Podoprikhin, T. Garipov, D. P. Vetrov, and A. G. Wilson. Averag- ing weights leads to wider optima and better generalization. In A. Globerson and R. Silva, editors, Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, UAI 2018, Monterey, California, USA, August 6-10, 2018, pages 876–885. AUAI Press, 2018. [30] X. Jia, Y. Zhang, B. Wu, K. Ma, J. Wang, and X. Cao. LAS-AT: adversarial training with learnable attack strategy. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 13388–13398. IEEE, 2022. [31] A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny im- ages. 2009. [32] A. Kurakin, I. J. Goodfellow, and S. Bengio. Adversarial machine learning at scale. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. [33] B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predic- tive uncertainty estimation using deep ensembles. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, edi- tors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6402–6413, 2017. [34] Y. Le and X. Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015. [35] H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein. Visualizing the loss land- scape of neural nets. In S. Bengio, H. M. Wallach, H. Larochelle, K. Grau- man, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 6391– 6401, 2018. [36] L. Li and M. W. Spratling. Data augmentation alone can improve adversarial training. CoRR, abs/2301.09879, 2023. [37] F. Liao, M. Liang, Y. Dong, T. Pang, X. Hu, and J. Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 1778–1787. Computer Vision Foundation / IEEE Computer Society, 2018. [38] I. Loshchilov and F. Hutter. SGDR: stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. [39] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. [40] R. Müller, S. Kornblith, and G. E. Hinton. When does label smoothing help? In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. B. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 4696–4705, 2019. [41] D. Paleka and A. Sanyal. A law of adversarial risk, interpolation, and label noise. CoRR, abs/2207.03933, 2022. [42] T. Pang, K. Xu, C. Du, N. Chen, and J. Zhu. Improving adversarial robustness via promoting ensemble diversity. In K. Chaudhuri and R. Salakhutdinov, edi- tors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 4970–4979. PMLR, 2019. [43] T. Pang, X. Yang, Y. Dong, H. Su, and J. Zhu. Bag of tricks for adversarial training. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. [44] Y. Qin, X. Wang, A. Beutel, and E. H. Chi. Improving calibration through the rela- tionship with adversarial robustness. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin,P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 14358–14369, 2021. [45] R. Rade and S. Moosavi-Dezfooli. Reducing excessive margin to achieve a better accuracy vs. robustness trade-off. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. [46] S. Rebuffi, S. Gowal, D. A. Calian, F. Stimberg, O. Wiles, and T. A. Mann. Fixing data augmentation to improve adversarial robustness. CoRR, abs/2103.01946, 2021. [47] S.-A. Rebuffi, S. Gowal, D. A. Calian, F. Stimberg, O. Wiles, and T. Mann. Data augmentation can improve robustness. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021. [48] L. Rice, E. Wong, and J. Z. Kolter. Overfitting in adversarially robust deep learn- ing. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 8093–8104. PMLR, 2020. [49] L. Schmidt, S. Santurkar, D. Tsipras, K. Talwar, and A. Madry. ally robust generalization requires more data. Adversari- In S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, pages 5019–5031, 2018. [50] V. Sehwag, S. Mahloujifar, T. Handina, S. Dai, C. Xiang, M. Chiang, and P. Mittal. Robust learning meets generative models: Can proxy distributions improve adversar-ial robustness? In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. [51] L. N. Smith and N. Topin. Super-convergence: Very fast training of residual net- works using large learning rates. CoRR, abs/1708.07120, 2017. [52] J. Snoek, Y. Ovadia, E. Fertig, B. Lakshminarayanan, S. Nowozin, D. Sculley, J. V. Dillon, J. Ren, and Z. Nado. Can you trust your model’s uncertainty? evaluat- ing predictive uncertainty under dataset shift. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. B. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, pages 13969–13980, 2019. [53] D. Stutz, M. Hein, and B. Schiele. Confidence-calibrated adversarial training: Gen- eralizing to unseen attacks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 9155–9166. PMLR, 2020. [54] D. Stutz, M. Hein, and B. Schiele. Relating adversarially robust generalization to flat minima. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 7787–7797. IEEE, 2021. [55] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In Y. Bengio and Y. LeCun, ed- itors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. [56] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 2818–2826. IEEE Computer Society, 2016. [57] A. Tarvainen and H. Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings. OpenReview.net, 2017. [58] A. Torralba, R. Fergus, and W. T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Trans. Pattern Anal. Mach. Intell., 30(11):1958–1970, 2008. [59] J. Uesato, B. O’Donoghue, P. Kohli, and A. van den Oord. Adversarial risk and the dangers of evaluating against weak attacks. In J. G. Dy and A. Krause, edi- tors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 5032–5041. PMLR, 2018. [60] E. Wong and J. Z. Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In J. G. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 5283–5292. PMLR, 2018. [61] D. Wu, S. Xia, and Y. Wang. Adversarial weight perturbation helps robust gener- alization. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, edi- tors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. [62] S. Yun, D. Han, S. Chun, S. J. Oh, Y. Yoo, and J. Choe. Cutmix: Regularization strategy to train strong classifiers with localizable features. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 6022–6031. IEEE, 2019. [63] S. Zagoruyko and N. Komodakis. Wide residual networks. In R. C. Wilson, E. R. Hancock, and W. A. P. Smith, editors, Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, September 19-22, 2016. BMVA Press, 2016. [64] H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan. Theo- retically principled trade-off between robustness and accuracy. In K. Chaudhuri and R. Salakhutdinov, editors, International conference on machine learning, pages 7472–7482. PMLR, 2019. [65] S. Zhang, H. Gao, T. Zhang, Y. Zhou, and Z. Wu. Alleviating robust overfitting of adversarial training with consistency regularization. CoRR, abs/2205.11744, 2022. [66] S. Zhao, J. Yu, Z. Sun, B. Zhang, and X. Wei. Enhanced accuracy and robustness via multi-teacher adversarial distillation. In S. Avidan, G. J. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, editors, Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part IV, volume 13664 of Lecture Notes in Computer Science, pages 585–602. Springer, 2022. [67] J. Zhu, J. Yao, B. Han, J. Zhang, T. Liu, G. Niu, J. Zhou, J. Xu, and H. Yang. Re- liable adversarial distillation with unreliable teachers. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. [68] B. Zi, S. Zhao, X. Ma, and Y. Jiang. Revisiting adversarial robustness distillation: Robust soft labels make student better. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 16423–16432. IEEE, 2021. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/87975 | - |
| dc.description.abstract | 在標準的對抗訓練中，模型以獨熱標籤 (one-hot label) 作為優化目標，當對抗式攻擊 (Adversarial Attack) 對資料造成的變異在事先定義可接受的範圍之內，模型都將以相同的標籤作為目標學習。也就是，一定變異內的資料都標上相同的標籤。然而，這種給予標籤的方式使模型忽略了對抗式攻擊產生的變異對資料所帶來的潛在分佈飄移 (distribution shift) 現象，因而導致過度擬合 (overfitting) 的問題出現在對抗式訓練中。為了解決這個問題並強化模型防禦對抗式攻擊，首先，我們先分析了強健模型的特徵，並發現了強健模型更傾向產生平滑且校正良好的輸出。基於這項觀察，我們提出了一個簡單但有效的方法「漸進式自蒸餾校正」Annealing Self-Distillation Rectification，ADR），生成對資料分佈改變更精確描述的軟標籤，對模型訓練提供更好的引導，以準確反應攻擊下的資料分佈偏移狀況。透過漸進式自蒸餾校正得到的校正分佈軟標籤 (rectified labels)，我們得以在沒有預先訓練模型 (pre-trained models) 和額外大量計算的情況下顯著提昇模型的強健性。此外，透過以校正後的軟標籤替換損失函數中的硬標籤，我們的方法可以很方便的與其他對抗式訓練的演算法結合。我們的實驗在廣泛的資料集與模型架構上都得到了強勁的表現，驗證了漸進式自蒸餾校正是個有效改善對抗式訓練，並防止過度擬合的方法。 | zh_TW |
| dc.description.abstract | In standard adversarial training, models are optimized to fit one-hot labels within allowable adversarial perturbation budgets. However, the ignorance of underlying distribution shifts brought by perturbations causes the problem of robust overfitting. To address this issue and enhance adversarial robustness, we analyze the characteristics of robust models and identify that robust models tend to produce smoother and well-calibrated outputs. Based on the observation, we propose a simple yet effective method, Annealing Self-Distillation Rectification (ADR), which generates soft labels as a better guidance mechanism that accurately reflects the distribution shift under attack during adversarial training. By utilizing ADR, we can obtain rectified distributions that significantly improve model robustness without the need for pre-trained models or extensive extra computation. Moreover, our method facilitates seamless plug-and-play integration with other adversarial training techniques by replacing the hard labels in their objectives. We demonstrate the efficacy of ADR through extensive experiments and strong performances across datasets. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2023-08-01T16:10:48Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2023-08-01T16:10:48Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | Verification Letter from the Oral Examination Committee i
Acknowledgements iii 摘要 v Abstract vii Contents ix List of Figures xi List of Tables xiii Denotation xv Chapter 1 Introduction 1 Chapter 2 Related Work 5 2.1 Robust Overfitting 5 2.2 Rectify labels in AT 6 Chapter 3 Preliminaries 9 3.1 Adversarial training (AT) 9 3.2 Distributional difference in the outputs of robust and non-robust model 10 3.2.1 Robust model generates a random output on OOD data 10 3.2.2 Robust models are uncertain on incorrectly classified examples 11 3.2.3 Output distribution of models on clean or adversarial examples are consistent 12 Chapter 4 Methodology 15 4.1 Motivation: Rectify labels in a noise-aware manner 15 4.2 Annealing Self-Distillation Rectification 16 Chapter 5 Experiments 19 5.1 Training and evaluation setup 19 5.2 Superior performance across robustified methods and datasets 21 5.3 Combing with weight space smoothing techniques and scaling to larger architecture 22 5.4 Test accuracy of TRADES + ADR combing with WA and AWP 23 5.5 Comparison with related works and use additional data on CIFAR-100 23 5.6 Test accuracy (%) compared with related works on TinyImageNet-200. 25 5.7 Achieving flatter weight loss landscape 25 5.8 Ablation study on the effectiveness of temperature and interpolation factor 26 5.9 Sanity check for gradient obfuscation 29 5.10 Computation cost analysis 30 5.11 Variance across reruns 31 Chapter 6 Conclusion 35 6.1 Limitations 35 6.2 Border impacts 36 6.3 Conclusion 36 References 39 | - |
| dc.language.iso | en | - |
| dc.subject | 過度擬合 | zh_TW |
| dc.subject | 對抗式訓練 | zh_TW |
| dc.subject | 知識蒸餾 | zh_TW |
| dc.subject | Adversarial Training | en |
| dc.subject | Robust Overfitting | en |
| dc.subject | Knowledge Distillation | en |
| dc.title | 利用漸進式自蒸餾方法改進對抗式訓練 | zh_TW |
| dc.title | Annealing Self-Distillation Rectification Improves Adversarial Training | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 111-2 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 陳祝嵩;曹昱 | zh_TW |
| dc.contributor.oralexamcommittee | Chu-Song Chen;Yu Tsao | en |
| dc.subject.keyword | 對抗式訓練,過度擬合,知識蒸餾, | zh_TW |
| dc.subject.keyword | Adversarial Training,Robust Overfitting,Knowledge Distillation, | en |
| dc.relation.page | 51 | - |
| dc.identifier.doi | 10.6342/NTU202301129 | - |
| dc.rights.note | 同意授權(全球公開) | - |
| dc.date.accepted | 2023-06-29 | - |
| dc.contributor.author-college | 電機資訊學院 | - |
| dc.contributor.author-dept | 資訊工程學系 | - |
| 顯示於系所單位： | 資訊工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-111-2.pdf | 3.17 MB | Adobe PDF | 檢視/開啟 |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。