請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/91756完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 林守德 | zh_TW |
| dc.contributor.advisor | Shou-De Lin | en |
| dc.contributor.author | 黃柏瑋 | zh_TW |
| dc.contributor.author | Bo-Wei Huang | en |
| dc.date.accessioned | 2024-02-22T16:34:49Z | - |
| dc.date.available | 2024-02-23 | - |
| dc.date.copyright | 2024-02-22 | - |
| dc.date.issued | 2023 | - |
| dc.date.submitted | 2023-07-10 | - |
| dc.identifier.citation | Robert Adragna, Elliot Creager, David Madras, and Richard S. Zemel. Fairness and robustness in invariant learning: A case study in toxicity classification. ArXiv, abs/2011.06485, 2020.
Bang An, Zora Che, Mucong Ding, and Furong Huang. Transferring fairness under distribution shifts via fair consistency regularization. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=zp_Cp38qJE0. Julia Angwin, Jeff Larson, Lauren Kirchner, and Surya Mattu. Machine bias: There＇s software used across the country to predict future criminals and it＇s biased against blacks., May 2016. URL https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization, 2019. URL https://arxiv.org/abs/1907.02893. Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018. Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. Fairlearn: A toolkit for assessing and improving fairness in AI. Technical Report MSR-TR-2020-32, Microsoft, May 2020. URL https://www.microsoft.com/en-us/research/publication/fairlearn-a-toolkit-for-assessing-and-improving-fairness-in-ai/. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal, sep 2015. Association for Computational Linguistics. doi: 10.18653/v1/D15-1075. URL https://aclanthology.org/D15-1075. Shiyu Chang, Yang Zhang, Mo Yu, and Tommi Jaakkola. Invariant rationalization. In Proceedings of the 37th International Conference on Machine Learning, pages 1448– 1458, 2020. URL https://proceedings.mlr.press/v119/chang20c.html. Yo Joong Choe, Jiyeon Ham, and Kyubyong Park. An empirical study of invariant risk minimization. In ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning, 2020. Moulik Choraria, Ibtihal Ferwana, Ankur Mani, and Lav R. Varshney. Balancing fairness and robustness via partial invariance. ArXiv, abs/2112.09346, 2021. Moulik Choraria, Ibtihal Ferwana, Ankur Mani, and Lav R. Varshney. Learning optimal features via partial invariance. ArXiv, abs/2301.12067, 2023. Elliot Creager, Joern-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning. In Proceedings of the 38th International Conference on Machine Learning, pages 2189–2200, 2021. URL https://proceedings.mlr.press/v139/creager21a.html. Enyan Dai and Suhang Wang. Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 2020. Yana Dranker, He He, and Yonatan Belinkov. IRM—when it works and when it doesn’t: A test case of natural language inference. In Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=KtvHbjCF4v. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, page 214–226, New York, NY, USA, 2012. URL https://doi.org/10.1145/2090236.2090255. Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020. URL https://doi.org/10.1038%2Fs42256-020-00257-z. Max Glockner, Vered Shwartz, and Yoav Goldberg. Breaking NLI systems with sentences that require simple lexical inferences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 650–655, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/P18-2103. URL https://aclanthology.org/P18-2103. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 107–112, June 2018. doi: 10.18653/v1/N18-2017. URL https://aclanthology.org/N18-2017. Sara Hajian and Josep Domingo-Ferrer. A methodology for direct and indirect discrimination prevention in data mining. IEEE Transactions on Knowledge and Data Engineering, 25(7):1445–1459, 2013. doi: 10.1109/TKDE.2012.72. Moritz Hardt, Eric Price, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, 2016. URL https://proceedings.neurips.cc/paper/2016/file/9d2682367c3935defcb1f9e247a97c0d-Paper.pdf. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, Los Alamitos, CA, USA, jun 2016. IEEE Computer Society. doi: 10.1109/CVPR.2016.90. URL https://doi.ieeecomputersociety.org/10.1109/CVPR.2016.90. Arthur Jacot, Clément Hongler, and Franck Gabriel. Neural tangent kernel: Convergence and generalization in neural networks. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada, pages 8580–8589, 2018. Ronny Kohavi and Barry Becker. Adult. UCI Machine Learning Repository, 1996. Accessed: 2021-09-06. David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapolation (rex). In Proceedings of the 38th International Conference on Machine Learning, pages 5815–5826, 2021. URL https://proceedings.mlr.press/v139/krueger21a.html. Kun Kuang, Ruoxuan Xiong, Peng Cui, Susan Athey, and Bo Li. Stable prediction with model misspecification and agnostic distribution shift. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4485–4492, 04 2020. doi: 10.1609/aaai.v34i04.5876. Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. In Advances in Neural Information Processing Systems, pages 728–740, 2020. URL https://proceedings.neurips.cc/paper/2020/file/07fc15c9d169ee48573edd749d25945d-Paper.pdf. Yong Lin, Shengyu Zhu, and Peng Cui. Zin: When and how to learn invariance by environment inference?, 2022. URL https://arxiv.org/abs/2203.05818. Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Proceedings of the 38th International Conference on Machine Learning, pages 6781–6792, 2021. URL https://proceedings.mlr.press/v139/liu21f.html. Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li, and Zheyan Shen. Heterogeneous risk minimization. In Proceedings of the 38th International Conference on Machine Learning, pages 6804–6814, 2021. URL https://proceedings.mlr.press/v139/liu21h.html. Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li, and Zheyan Shen. Kernelized heterogeneous risk minimization. In Advances in Neural Information Processing Systems, pages 21720–21731, 2021. URL https://proceedings.neurips.cc/paper/2021/file/b59a51a3c0bf9c5228fde841714f523a-Paper.pdf. Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. ArXiv, abs/1705.07874, 2017. Tom McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428– 3448, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1334. URL https://aclanthology.org/P19-1334. Ninareh Mehrabi, Fred Morstatter, Nripsuta Ani Saxena, Kristina Lerman, and A. G. Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54:1 – 35, 2019. Sungho Park, Sunhee Hwang, Dohyung Kim, and Hyeran Byun. Learning disentangled representation for fair facial attribute classification via fairness-aware information alignment. In AAAI Conference on Artificial Intelligence, 2021. Judea Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988. J. Peters, Peter Buhlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78, 2015. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. Hypothesis only baselines in natural language inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 180–191, New Orleans, Louisiana, jun 2018. Association for Computational Linguistics. doi: 10.18653/v1/S18-2023. URL https://aclanthology.org/S18-2023. Yao Qiang, Chengyin Li, Prashant Khanduri, and D. Zhu. Fairness-aware vision transformer via debiased self-attention. ArXiv, abs/2301.13803, 2023. John Rawls. Justice as Fairness: A Restatement. Harvard University Press, 2001. Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. Improving fair training under correlation shifts. ArXiv, abs/2302.02323, 2023. Mateo Rojas-Carulla, Bernhard Schölkopf, Richard E. Turner, and J. Peters. Invariant models for causal transfer learning. J. Mach. Learn. Res., 19:36:1–36:34, 2018. Shiori Sagawa*, Pang Wei Koh*, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=ryxGuJrFvS. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS 2019 Workshop on Energy Efficient Machine Learning and Cognitive Computing, 2019. Harini Suresh and John V. Guttag. A framework for understanding sources of harm throughout the machine learning life cycle. Equity and Access in Algorithms, Mechanisms, and Optimization, 2019. Tianlu Wang, Diyi Yang, and Xuezhi Wang. Identifying and mitigating spurious correlations for improving robustness in nlp models. In NAACL-HLT, 2021. P Welinder, S Branson, T Mita, C Wah, F Schroff, S Belongie, and P Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, Caltech, 2010. Michael Wick, swetasudha panda, and Jean-Baptiste Tristan. Unlocking fairness: a trade-off revisited. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/373e4c5d8edfa8b74fd4b6791d0cf6dc-Paper.pdf. Linda F. Wightman. Lsac national longitudinal bar passage study. lsac research report series. 1998. Depeng Xu, Yongkai Wu, Shuhan Yuan, Lu Zhang, and Xintao Wu. Achieving causal fairness through generative adversarial networks. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 1452–1458. International Joint Conferences on Artificial Intelligence Organization, 7 2019. doi: 10.24963/ijcai.2019/201. URL https://doi.org/10.24963/ijcai.2019/201. Shen Yan, Hsien-te Kao, and Emilio Ferrara. Fair class balancing: Enhancing model fairness without observing sensitive attributes. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, page 1715–1724, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450368599. doi: 10.1145/3340531.3411980. URL https://doi.org/10.1145/3340531.3411980. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P. Gummadi. Fairness Constraints: Mechanisms for Fair Classification. In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 962–970. PMLR, 20–22 Apr 2017. URL https://proceedings.mlr.press/v54/zafar17a.html. Fengda Zhang, Kun Kuang, Long Chen, Yuxuan Liu, Chao Wu, and Jun Xiao. Fairness-aware contrastive learning with partially annotated sensitive attributes. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=woa783QMul. Lu Zhang, Yongkai Wu, and Xintao Wu. A causal framework for discovering and removing direct and indirect discrimination. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 3929– 3935, 2017. doi: 10.24963/ijcai.2017/549. URL https://doi.org/10.24963/ijcai.2017/549. Tianxiang Zhao, Enyan Dai, Kai Shu, and Suhang Wang. Towards fair classifiers without sensitive attributes: Exploring biases in related features. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 2021. Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6):1452–1464, 2018. doi: 10.1109/TPAMI.2017.2723009. Chunting Zhou, Xuezhe Ma, Paul Michel, and Graham Neubig. Examining and combating spurious features under distribution shift. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 12857– 12867. PMLR, 18–24 Jul 2021. URL https://proceedings.mlr.press/v139/zhou21g.html. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/91756 | - |
| dc.description.abstract | 神經網路經常透過「經驗風險最小化」進行訓練；然而，近來許多研究表明模型可能無意間學習到資料中的偏差，影響分佈外泛化能力和演算法公平性。為了解決這個問題，「不變學習」被提出用於提取不受分佈變化影響的不變特徵，而本研究提出了一個不變學習框架 EDNIL，其包含多頭神經網路以吸收資料中的偏差。在論文中，我們說明此框架不需要事先取得環境標籤，也沒有過度要求預訓練模型。此外，我們揭示了此演算法與近期討論變異特徵和不變特徵屬性研究之間的理論連結。最後，透過實驗證明，使用EDNIL訓練的模型不僅表現出卓越的分佈外泛化能力，還能促進機器學習演算法的公平性。 | zh_TW |
| dc.description.abstract | Neural networks are often trained with Empirical Risk Minimization (ERM). However, it has been shown that model could inadvertently learn biases present in data, causing out-of-distribution generalization and algorithmic fairness concerns. On this issue, a research direction, invariant learning, has been proposed to extract invariant features insensitive to the distributional changes. This work proposes EDNIL, an invariant learning framework containing an auxiliary multi-head neural network to absorb data biases. We show that this framework does not require prior knowledge about environments or strong assumptions about the pre-trained model. We also reveal that the proposed algorithm has theoretical connections to recent studies discussing properties of variant and invariant features. Finally, we empirically demonstrate that models trained with EDNIL not only exhibit enhanced out-of-distribution generalization but also promote algorithmic fairness. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2024-02-22T16:34:49Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2024-02-22T16:34:49Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | 口試委員會審定書 i
誌謝 ii 摘要 iii Abstract iv Contents v List of Figures viii List of Tables ix Chapter 1 Introduction 1 Chapter 2 Preliminaries and Related Work 5 2.1 Out-of-Distribution Generalization and Invariant Learning 5 2.2 Ideal Environments 7 2.3 Unsupervised Environment Inference 7 2.4 Algorithmic Fairness 9 Chapter 3 Methodology 11 3.1 The Environment Inference Model 12 3.1.1 Inference Stage of MEI 13 3.1.2 Learning Stage of MEI 13 3.2 The Invariant Learning Model 15 Chapter 4 Experiments for Out-of-Distribution Generalization 17 4.1 Simple Classification with MLP 18 4.1.1 Discussions on Adult-Confounded 18 4.1.1.1 Main Results 20 4.1.1.2 Ablation Study for LEI 20 4.1.2 Discussions on CMNIST 21 4.1.2.1 Main Results 22 4.1.2.2 Different Strengths of Spurious Correlation 22 4.2 Complex Classification with Pre-trained Models 24 4.2.1 Discussions on Waterbirds 24 4.2.1.1 Main Results 25 4.2.1.2 Choice of Initialization 26 4.2.2 Discussions on SNLI 27 4.2.2.1 Main Results 28 4.3 Extension to Regression 28 Chapter 5 Experiments for Algorithmic Fairness 31 5.1 Evaluation on Performance and Fairness Metrics 32 5.2 Evidence of Diversified Sensitive Information 35 Chapter 6 Conclusion 37 References 39 Appendix A — Theoretical Proof 49 A.1 The Underlying Graphical Model for EDNIL 49 Appendix B — Experimental Details 50 B.1 Implementation Resources 50 B.2 Hyper-parameter Tuning 51 | - |
| dc.language.iso | en | - |
| dc.subject | 分佈外泛化 | zh_TW |
| dc.subject | 不變學習 | zh_TW |
| dc.subject | 機器學習 | zh_TW |
| dc.subject | 深度學習 | zh_TW |
| dc.subject | 演算法公平性 | zh_TW |
| dc.subject | Algorithmic Fairness | en |
| dc.subject | Deep Learning | en |
| dc.subject | Machine Learning | en |
| dc.subject | Out-of-Distribution Generalization | en |
| dc.subject | Invariant Learning | en |
| dc.title | 利用無監督式環境多樣化解決分佈外泛化與演算法公平性之問題 | zh_TW |
| dc.title | Unsupervised Environment Diversification for Out-of-Distribution Generalization and Algorithmic Fairness | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 112-1 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 陳縕儂;陳祝嵩;孫紹華;鄭卜壬 | zh_TW |
| dc.contributor.oralexamcommittee | Yun-Nung Chen;Chu-Song Chen;Shao-Hua Sun;Pu-Jen Cheng | en |
| dc.subject.keyword | 不變學習,分佈外泛化,演算法公平性,深度學習,機器學習, | zh_TW |
| dc.subject.keyword | Invariant Learning,Out-of-Distribution Generalization,Algorithmic Fairness,Deep Learning,Machine Learning, | en |
| dc.relation.page | 51 | - |
| dc.identifier.doi | 10.6342/NTU202301438 | - |
| dc.rights.note | 同意授權(全球公開) | - |
| dc.date.accepted | 2023-07-12 | - |
| dc.contributor.author-college | 電機資訊學院 | - |
| dc.contributor.author-dept | 資訊工程學系 | - |
| 顯示於系所單位： | 資訊工程學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-112-1.pdf | 2.45 MB | Adobe PDF | 檢視/開啟 |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。