停止說‘Delve’! 通過復合分佈對齊適配大型語言模型以進行文本風格化

Abstract

自 ChatGPT 推出以來，使用者——尤其是非英語母語者——使用大型語言模型（LLMs）提供的服務，有效地幫助了他們表達想法和生產內容。然而，近期有觀察到大型語言模型使用率正急劇上升，也暴露了其輸出文本具有可識別的寫作風格，進而讓使用者沒有辦法去有效提高生產力且被污名化。為了要去處理這個問題，重要的地方在於使大型語言模型所產出的內容與人類撰寫的文本相似。本次的研究中，我們提出複合分佈對齊（Composite Distributional Alignment，簡稱 CoDA），其中包括零階偏差對齊啟發式算法（Zeroth-order Bias Alignment Heuristics，簡稱 ZoBAH）和判別器自舉提名（Discriminator Bootstrapped Nomination，簡稱 DiBoN）。CoDA 通過在 ZoBAH 中以正反文本的方法篩選文本中具有偏差的字符並給予校正，和在 DiBoN 中針對動態分數重新調整大型語言模型產出的字符流程。具體來說，ZoBAH 解決了大型語言模型產出與專家文本之間在詞彙層面的統計差異，而 DiBoN 進一步結合現成的 AI 檢測器、句法和語義特徵，考慮了更廣範圍的文本差異。 我們在 Multi-XScience 和 BAWE 資料集上的實驗，證實了 CoDA 的可行性。在白盒和黑盒場景中，它在傳統的字詞上、句型和文義檢測上取得了大量的改進。與現有最好的標準方法相比，將最先進 AI 檢測器的檢測率在白盒場景中降低了近 20%。另外，這個研究也展示了 CoDA 的可轉移性，展示了其構建通用權重的潛力，有效地消除了字詞、句型和文義特徵層面的誤差。

After the release of ChatGPT, large language models (LLMs) have provided significant assistance to non-native English speakers by refining their scientific writings. However, these models often generate text with a distinctive style that could potentially stigmatize its users. To mitigate this effect, it is essential to tailor LLM-generated content to more closely mimic human-produced texts. In this work, we present Composite Distributional Alignment (CoDA), which includes Zeroth-order Bias Alignment Heuristics (ZoBAH) and Discriminator Bootstrapped Nomination (DiBoN). CoDA modifies the autoregressive token generation in LLMs by adjusting logits using static biases from in data-driven fashion in ZoBAH and dynamic scores from DiBoN for top token options. Specifically, ZoBAH addresses word-level statistical disparities between LLM outputs and expert texts, while DiBoN further adjusts pattern-level criteria, incorporating off-the-shelf AI detectors, syntactic, and semantic features. Our extensive tests on both Multi-XScience and BAWE datasets confirm that CoDA significantly outperforms existing methods according to standard word- and pattern-level metrics under both white-box and black-box conditions, achieving up to 15% reduction in detection rates by advanced AI detectors compared with the strongest baseline in white-box setups. Furthermore, our studies reveal that CoDA is effective not only in adjusting biases but also in transferring knowledge across different contexts, thereby improving overall text quality, which suggests its utility in developing a universal weight capable of mitigating biases effectively at multiple levels.