人類與GPT的純協調遊戲

Abstract

我們在Prolific上舉行了一場全球的純協調遊戲實驗，其中包含四組參與者：與人類協調的人類（HH組）、與ChatGPT協調的人類（HG組）、與人類協調的ChatGPT（GH組）、與ChatGPT協調的ChatGPT（GG組）。比較四組的回答相似度，我們驚訝地發現相對於HH組，HG組的回答反而比較不像GH組。我們事前請了另外一小批參與者為我們的純協調遊戲題目標記出「可用性」最高的選項（A選項）與最不一樣的選項（O選項）。我們發現ChatGPT極度傾向於選擇A選項。HH組與HG組也傾向於選擇A選項，但相較之下HG組選了較多O選項，這正是前述發現的主因。結合參與者在李克特量表自評的直覺/思考傾向、每題所花秒數、參與時當地時間，我們擬合了兩個logistic generalized linear mixed models。我們認為，對HH組而言直覺的選擇是A選項，對HG組而言直覺的選擇是O選項。我們還測試並整理了人類、ChatGPT、Claude、Llama三種模型在五種純協調遊戲中的決策。

We conducted a global pure coordination game experiment on Prolific with four participant groups: humans coordinating with humans (HH), humans coordinating with ChatGPT (HG), ChatGPT coordinating with humans (GH), and ChatGPT coordinating with ChatGPT (GG). Surprisingly, we found that compared to the HH group, the responses of the HG group were actually less similar to those of the GH group. We asked a separate group of participants to label the option with the highest "availability" (A option) and the odd-one-out (O option) for each coordination item. We found that ChatGPT participants had a strong tendency to choose A options, but HG participants selected more O options than HH participants, which mainly explains the previous finding. Using participants' intuition-thinking tendency, response time per trial, and local time of participation, we fit two logistic generalized linear mixed models. Based on the results, we argue that for the HH group, A was the intuitive choice, while for the HG group, O was the intuitive choice.