透過微調技術、對比學習、全面訓練策略和實際評估增進程式合成

Abstract

程式合成基於特定的規格來創建程式，這些規格可以有各種形式。大型語言模型（LLM）由於缺乏訓練資料，在處理領域特定語言（DSL）時存在困難。了解DSL與一般程式語言之間的差異至關重要。我們開發了兩個框架來改進模型對DSL執行和邊緣情況的理解。此外，添加新的神經模塊也可能有幫助。我們利用參數高效微調（PEFT）和CLIP開發了具有增強泛化能力的兩個框架。在某些情況下，設計評估指標可能是必要的。我們的貢獻在於找出最有效的方法來彌合DSL與LLM之間的鴻溝，並通過使用新的評估指標，提供對神經程式合成的新視角。

Program synthesis creates programs based on specific specifications in various modalities. Large language models~(LLMs) struggle with domain-specific language~(DSL) due to a lack of training data. Understanding the differences between DSL and general programming languages is important. Two frameworks have been developed to improve the model''s understanding of DSL execution and corner cases. Adding new neural modules may also help. Two frameworks with enhanced generalization abilities have been developed using parameter-efficient fine-tuning (PEFT) and CLIP. In some cases, designing an evaluation metric may be necessary. Our contribution involves identifying the most effective method for bridging the gap between DSL and LLM and offering a fresh perspective on neural program synthesis through the use of new evaluation metrics.