透過 CLIP 文字編碼器適應進行條件擴散模型中的語義編輯和去偏見

Abstract

本論文研究了 CLIP 文本編碼器在條件擴散模型中的適應，以解決語義編輯和去偏見相關的挑戰。我們探討了透過適應 (Adaptation) 在增強生成圖像語義屬性控制方面的有效性，同時減少內在偏見。研究利用了各種解耦策略，並對文字編碼器進行修改，以評估緩解與性別和種族相關偏見的潛力。我們的研究結果表明，通過針對性適應微調文本編碼器可以顯著提高語義控制的精確性和去偏見的有效性。本研究為圖像合成領域的更公平和可控的生成模型的發展做出了貢獻。

This thesis investigates the adaptation of the CLIP text encoder for use in conditionaldiffusion models to address challenges related to semantic editing and debiasing. We explore the effectiveness of low-rank adaptations in enhancing the control over semanticattributes of generated images while simultaneously reducing inherent biases. The studyutilizes various disentanglement strategies and introduces modifications to the text encoder to evaluate the potential for mitigating biases related to gender and ethnicity. Ourfindings indicate that fine-tuning the text encoder with targeted adaptations can significantly improve semantic control’s precision and debiasing effectiveness. This work contributes to the development of more fair and controllable generative models in the field ofimage synthesis.