匯總語義匹配檢索：透過開放式問題回答來釋放大型語言模型的能力

Abstract

雖然大型語言模型（LLMs）在許多自然語言處理任務中展示了出色的最新技術（SOTA）表現，但多選問答任務對於大型語言模型來說，仍然是一個挑戰。儘管通過利用少量樣本上下文學習，大型語言模型在多選問答任務上取得了不錯的成果。生成或選擇示範樣本並不是一項簡單的工作。在示範樣本的設計中必須考慮到許多因素，包括它們的順序、主要標籤和最近的標籤。最近有幾項研究專注於示範樣本的設計，以期望能達到更好的性能。因此，我們想知道如果不考慮示範樣本，我們如何通過零樣本學習來提高大型語言模型在多選推理任務中的性能？ 當人們面臨多選推理任務時，我們通常依賴我們的先驗知識和常識來形成初步的答案。隨後，將這個初步答案與提供的選項進行比較，並選擇最有可能的選項作為最終答案。因此，我們提出聚合語義匹配檢索（ASMR）作為多選推理任務的解決方案。為了模仿人類解決多選推理任務的過程，我們利用大型語言模型的能力，首先通過開放式問題生成初步可能的答案，透過比較這個初步的答案與提供的選項，如此有助於從給定的選項中檢索出更有可能的答案選項。我們的實驗表明，ASMR在流行的常識推理資料集以及BIG-BENCH資料集上取得了良好的成果。

While Large Language Models (LLMs) have showcased remarkable state-of-the-art (SOTA) performance across numerous natural language processing tasks, their effectiveness in multiple-choice question answering tasks continues to pose a challenge. Although by utilizing few-shot in-context learning, LLMs have chieved impressive results on multiple choice question answering tasks. To generate or select of demonstration samples is not a simple work. Several factors must be taken into account during the design of demonstration samples, including their sequence, predominant label, and recentness label. There are several studies recently focus on the design of demonstration samples to achieve better performance. Therefore, We are wondering that without considering demonstration samples, how can we enhance the performance of Large Language Models on multiple-choice reasoning tasks through zero-shot learning? When confronted with multiple-choice reasoning tasks, humans typically rely on their prior knowledge and commonsense to formulate a preliminary answer in mind. Subsequently, they compare this preliminary answer to the provided choices, and select the most likely choice as the final answer.We introduce Aggregated Semantic Matching Retrieval (ASMR) as a solution for multiple-choice reasoning tasks. To mimic the process of humans solving reasoning tasks with multiple choices, we leverage the capabilities of LLMs to first generate the preliminary possible answers through open-ended question which aids in enhancing the process of retrieving relevant answers to the question from the given choices. Our experiments demonstrate the effectiveness of ASMR on popular commonsense reasoning benchmark and BIG-BENCH datasets.