基於候選篩選與噪音注入的動態檢索增強生成架構

Abstract

檢索增強生成 (Retrieval-Augmented Generation, RAG) 通過提供外部知識庫，大幅增強了大型語言模型的能力。然而，傳統的架構通常會檢索固定數量的相關文章，導致實作時出現一些問題：過少的文章可能會遺漏關鍵資訊，而過多的文章則可能引入干擾文章並使大型語言模型的輸入過長。為了解決這些問題，我們提出了一個全新的框架，此框架能夠實現動態的外部知識文章組成。首先，透過實施候選篩選模組來動態篩選檢索到的候選文章集合。再來，受先前研究的啟發，我們引入了噪音注入模組，通過系統性地加入隨機文章，來形成最終的參考文章集合。我們在六個知名的資料集上進行了實驗，結果發現，我們提出的框架在準確性方面持續優於傳統的架構。我們的研究結果突顯了動態的外部知識文章組成的優勢，並為構建更有效的 RAG 系統提供了實用的解決方案。

Retrieval-Augmented Generation (RAG) significantly enhances Large Language Models (LLMs) by providing external knowledge base. However, traditional RAG typically retrieve fixed number (top-k) of articles, often struggle with some problems: too few articles may miss crucial information, while too many can introduce distractors and overwhelm the LLM. To address these limitations, we propose a novel framework that enables an adaptive reference articles formulation of RAG. Our approach moves beyond the fixed top-k paradigm by implementing candidate pruning module to dynamically filtering a retrieved candidate articles set. Furthermore, inspired by prior work, we introduce another module for noise injection through the systematic inclusion of random articles. We conduct experiments on six well-known datasets, demonstrating that our proposed framework consistently outperforms traditional top-k RAG baselines in terms of generation accuracy. Our findings highlight the significant benefits of adaptive context formulation and offer practical solution for building more effective RAG systems.