利用檢索增強推理解決註冊會計師考試

Abstract

人工智慧中大型語言模型（LLM）的使用激增凸顯了它們在文字處理和生成方面的先進能力。然而，它們在會計和金融等專業領域的熟練程度仍然受到審查，特別是在註冊會計師（CPA）考試等複雜任務方面。在美國，CPA考試由美國註冊會計師協會（AICPA）監督，包括四個部分：審計和簽證（AUD）、商業環境與理論（BEC）、財務會計和報告（FAR）以及法規（REG）。過往研究表明，包括ChatGPT在內的LLM在CPA考試中遇到了複雜的問題解決場景和多樣化的問題類型，這表明需要進一步改進才能有效地處理此類特定領域的任務。 為了解決CPA考試的挑戰，引入了一種稱為檢索增強推理（RAR）的新方法，將平均通過率從0.5提高到0.62。RAR使用任務路由器將問題分為知識密集和推理密集型類別。對於知識密集型問題，它使用檢索增強生成（RAG）從外部資料庫中提取相關信息，以提高答案準確性。對於推理密集問題，RAR採用推理行動（ReAct）、代理人（Agent）和思想鏈（CoT）方法，並整合會計Python庫等外部工具，模仿真實考試環境，有效解決複雜問題。

The surge in the use of Large Language Models (LLMs) in artificial intelligence highlights their advanced capabilities in text processing and generation. However, their proficiency in specialized fields, such as accounting and finance, remains under scrutiny, particularly regarding complex tasks like the Certified Public Accountant (CPA) examination. The CPA exam, overseen by the American Institute of CPAs, encompasses four sections: Auditing and Attestation (AUD), Business Environment and Concepts (BEC), Financial Accounting and Reporting (FAR), and Regulation (REG). Research indicates that LLMs, including ChatGPT, struggle with the exam's complex problem-solving scenarios and varied question types, demonstrating the need for further improvement to handle such domain-specific tasks effectively. To address the challenges of the CPA exam, a new method called Retrieval Augmented Reasoning (RAR) has been introduced, improving the average pass rate from 0.5 to 0.62. RAR employs a task router to classify questions into knowledge-intensive and reasoning-intensive categories. For knowledge-intensive questions, it uses Retrieval Augmented Generation (RAG) to extract relevant information from external databases, enhancing answer accuracy. For reasoning-intensive questions, RAR utilizes ReAct, Agent, and Chain of Thought (CoT) approach, and integrates external tools like the accounting Python library to solve complex problems effectively, mimicking the real exam environment.