LLM誘導能力與衡量: 以電話訪問為例

Abstract

隨著大型語言模型（Large Language Models, LLMs）技術的迅速發展，人工智慧在語音辨識、自然語言處理與文本生成等領域已展現出高度成熟的應用潛力。特別是以ChatGPT為代表的語言模型，其高度擬人化的語言理解與生成能力，已逐漸成為商業實務與研究實驗中的關鍵工具。近年來的研究指出，LLM不僅能生成流暢的自然語言文本，亦展現出一定程度的說服能力，能在特定議題上影響人類使用者的立場。然而，現有文獻大多集中於單輪互動情境，對於多輪對話中LLM是否能展現持續而有效的誘導能力，尚缺乏系統性探討。

本研究以結構化的電話訪談作為實驗框架，透過Prompt Engineering技術，設計出具誘導策略的提問方式，模擬LLM作為訪問者（Interviewer），並以另一個LLM扮演受訪者（Interviewee），以多輪對話的形式觀察立場變化。研究設計中將逐輪紀錄受訪者的立場分數變動，並分析誘導性問題對於受訪者的影響。研究將從多個面向進行分析，包括不同主題下的立場變化趨勢、誘導策略的組合效果差異，LLM的誘導能力以及模型間的比較。

綜合而言，本研究不僅驗證了LLM於多輪對話中誘導能力的可行性，也比較了不同模型、主題與策略組合在實際對話中的差異，為理解與規範AI語言互動帶來新的實證依據，並呼籲對誘導性對話可能帶來之倫理風險與使用規範進行更全面的關注與討論。

With the rapid advancement of Large Language Models (LLMs), artificial intelligence has demonstrated significant potential in fields such as speech recognition, natural language processing, and text generation. Among these, language models like ChatGPT have become essential tools in both industrial applications and academic research, owing to their highly human-like capabilities in language understanding and generation. Recent studies have shown that LLMs not only produce fluent natural language output but also exhibit a certain degree of persuasive power, influencing users’ stances on specific issues. However, most existing research focuses on single-turn interactions, leaving the question of whether LLMs can exert sustained and effective persuasive influence in multi-turn dialogues largely unexplored.

This study adopts a structured telephone interview setting as the experimental framework. By leveraging Prompt Engineering, we design questions with embedded persuasive strategies, using one LLM to simulate the role of the interviewer and another to act as the interviewee. The experiment observes how the interviewee’s stance evolves across multiple dialogue turns. Each round records the interviewee’s stance score, and the effects of persuasive questioning are analyzed accordingly. The study takes a multi-faceted approach, examining stance change trends across different topics, the effects of various strategy combinations, the overall persuasive capacity of LLMs, and cross-model comparisons.

In summary, this research not only verifies the feasibility of LLMs exerting persuasive influence in multi-turn dialogues, but also compares how different models, topics, and strategy combinations behave in realistic dialogue settings. The findings provide empirical evidence for understanding and regulating AI-mediated language interactions and call for greater attention to the ethical risks and governance of persuasive AI dialogues.