行為提醒優於深度提問：在大型語言模型模擬中抑制健康假新聞的輕量化策略

Abstract

本研究將大型語言模型（LLM）視為模擬閱聽者，探討不同對話提示是否能提升其對健康新聞真偽判斷的準確性，並降低其轉傳意願。設計的提示包含蘇格拉底式提問，促使LLM反思，以及來自行為經濟學的簡單行為提醒(nudge)，用以警示轉傳假訊息的後果。實驗採用六篇經查證的長篇假新聞與六篇真新聞（來自權威機構），並依Big-Five理論建構十種典型人格與五種高風險人格。 本研究的輕量化干預分為兩種：(1) 蘇格拉底式提問，包含五題深度版與三題簡易版；(2) 行為提醒，內容為「轉傳未經查證的新聞可能造成誤導與社會恐慌，請謹慎評估」。 實驗採3 × 4因子設計（蘇格拉底提問 × nudge時機，共8組），以具長文本處理能力之GPT-4.1模擬閱聽者，針對六則假新聞與六則真新聞，結合十五種人格設定，收集初始與最終的判斷及轉傳意願，總計完成1440條對話紀錄。針對假新聞而言，結果顯示，LLM在基線下已達96%真偽判斷正確率，因此蘇格拉底式提問與nudge對判斷準確性無顯著提升，顯示具長文本處理能力之LLM已具備判斷新聞真偽的能力。然而，nudge明顯降低了轉傳意願：僅0.3%對話由「不轉傳」變為「轉傳」，9.6%則反向改變。合併有無nudge組，轉傳意願下降率分別為14.6%與0.8%；僅施以蘇格拉底提問則無顯著效果。 進一步分析發現，nudge出現於蘇格拉底提問之前或之後，對降低假新聞轉傳意願並無顯著差異，顯示只要給予簡潔的行為提醒，不論在提示流程中的順序如何，都能有效抑制轉傳意圖。 值得注意的是，本研究亦比較了干預措施對假新聞與真新聞的影響。結果顯示，無論是行為提醒還是蘇格拉底提問，都不僅有效抑制假新聞的轉傳意願，也會顯著降低真新聞的轉傳行為。在部分條件下，對真新聞的抑制效果甚至高於假新聞，突顯干預措施存在兩難：在遏止假訊息的同時，也可能抑制正確資訊流通。這一結果提醒，未來設計干預策略時，必須更加精細並兼顧情境，平衡抑制假新聞與促進可信內容的需求，特別是在公共衛生領域。 總結而言，對於模擬閱聽者的LLM而言，簡短的行為提醒比高成本的深度提問更有效且節省資源，能抑制健康假新聞的擴散。此輕量化策略適合大規模平台應用於假訊息防治，但其對真新聞的普遍抑制效應亦值得進一步探討。

This study treats large language models (LLMs) as simulated readers to examine whether different dialogue prompts can improve their accuracy in judging the veracity of health news — including both fake and true news — and reduce their willingness to share such news. The prompts include the Socratic questioning method to guide LLMs to reflect, and a simple behavioral nudge from behavioral economics to remind LLMs of the consequences of sharing misinformation. For the experiment, we used six long fact-checked fake news articles and six true news articles (sourced from authoritative organizations). We also built ten typical personalities based on the Big Five theory and five high-risk personalities. There are two main types of lightweight interventions: (1) Socratic questioning, which guides the model to reflect and includes a deep version with five questions and a simple version with three questions; (2) a behavioral nudge, which is a single reminder such as “Sharing unchecked news may mislead others and cause social panic. Please think carefully before sharing.” The experiment adopted a 3 × 4 factorial design (Socratic questioning × nudge timing, 8 groups in total), using GPT-4.1 with long-text processing capabilities to simulate readers. For six fake news and six true news articles, combined with fifteen personality profiles, both initial and final judgments and sharing intentions were collected, resulting in a total of 1,440 dialogue sessions. For fake news, the results show that LLMs already reached 96% accuracy in veracity judgment at baseline, so neither Socratic questioning nor the nudge significantly improved accuracy, suggesting that current LLMs with long text processing abilities can already judge news veracity well. However, the nudge clearly reduced willingness to share: only 0.3% of conversations changed from “not sharing” to “sharing,” while 9.6% changed in the opposite direction. When combining groups into “with nudge” and “without nudge,” the decrease in sharing intention was 14.6% and 0.8%, respectively; Socratic questioning alone had no significant effect. Further analysis revealed that the timing of the nudge—whether before or after Socratic questioning—did not significantly affect its ability to reduce sharing of fake news. This indicates that a single, concise behavioral reminder can effectively suppress sharing intention, regardless of placement within the prompt sequence. Notably, this study also compared the effects of interventions on both fake news and true news. Results show that while behavioral nudges and Socratic questioning both suppressed sharing of fake news, these interventions also significantly reduced the willingness to share true news. In some conditions, the reduction in sharing intention for true news was even greater than for fake news, highlighting a critical trade-off: interventions that curb misinformation may also inadvertently suppress accurate information. This comparative finding underscores the importance of developing nuanced and context-aware intervention strategies that balance suppression of fake news with the promotion of trustworthy content, especially in public health domains. In conclusion, for LLMs as simulated readers, a brief behavioral reminder is more effective and efficient than costly deep questioning in curbing the spread of health fake news. This lightweight approach offers practical value for scalable deployment on digital platforms to combat misinformation, but its broad suppressive effect on true news warrants further consideration and research.