資訊檢索與語言模型的協同效應：從資料高效密集檢索到事實性優 化

Abstract

近年來，人工智慧應用程式越來越依賴大量的一般和專門知識庫來處理複雜的任務，如問答和資訊搜尋。傳統的資訊檢索（IR）技術在有效地從各種知識來源收集相關資訊方面發揮了關鍵作用，從而提高了這些應用程式的性能。儘管這些資訊檢索系統十分有效，但它們需要大量的訓練資料，這限制了它們在各種任務中的應用。另一方面，大型語言模型（LLMs）因其能夠儲存在廣泛預訓練過程中獲得的大量知識而廣受歡迎。此外，這些模型擅長遵循各種指令並執行廣泛的任務。然而，大型語言模型在事實準確性方面存在困難，且無法提供最新資訊，這大大限制了它們成為下一代資訊存取引擎的效能。

因此，本篇論文旨在探索並增強資訊檢索和大型語言模型之間的協同效應，以提高它們的相互效能。具體來說，我們提出了資料高效檢索和大型語言模型的事實性對齊的技術。對於資料高效檢索，我們提出了在各個方面解決檢索系統資料效率的技術。通過利用大型語言模型的強大能力，我們提出了無需或只需少量標註資料即可建立檢索系統的新方法。對於事實性對齊，我們提出了一種對齊演算法 FactAlign，該演算法利用長篇事實性評估器提供的細粒度訊號。我們的研究重點為開發利用資訊檢索和大型語言模型優勢的技術，最終相互提高它們的效能。本篇論文中，每一篇研究都展示了一個用有效利用此協同效應的學習框架，總合起來，這篇論文探索了有效利用資訊檢索和大型語言模型之間的協同效應而相互提升彼此的效能。

In recent years, artificial intelligence applications have increasingly relied on substantial knowledge bases, both general and domain-specific, to address complex tasks such as question answering and information seeking. Traditional information retrieval (IR) techniques have played a critical role in enhancing the performance of these applications by effectively gathering relevant information from diverse knowledge sources. Despite their utility, these IR systems require extensive amount of training data, which hinders their application to various tasks.

On the other hand, large language models (LLMs) have gained immense popularity due to their ability to store vast amounts of knowledge acquired during extensive pretraining. In addition, these models excel in following diverse instructions and performing a wide array of tasks. However, LLMs struggle with factual accuracy and incapable of providing up-to-date information, which significantly limit their effectiveness to become the next-generation engine for information access.

In this thesis, we aim to explore and enhance the synergies between IR and LLMs, seeking to improve their mutual effectiveness. Specifically, we present techniques for data-efficient retrieval and factuality alignment of LLMs. For data-efficient retrieval, we present techniques that address the data efficiency of retrieval systems in various aspects. By leveraging the robust capabilities of LLMs, we propose novel methodologies to build retrieval systems with no or little annotated data. For factuality alignment, we propose an alignment algorithm, FactAlign, that leverages the fine-grained signals provided by long-form factuality evaluators. In summary, this thesis introduces various techniques that leverage the strengths of both IR and LLMs, ultimately improving their effectiveness mutually.