NEB-San: 全自動錯誤導向斷言生成插入器

Abstract

我們傳統上，會透過單元測試、模糊測試等方式來尋找程式的錯誤。單元測試擅於發現預定義問題；模糊測試擅長透過隨機產生的輸入值來快速執行程式，進而找到程式發生例外的問題。然而，若開發人員若沒有在單元測試預想到該測項，或者是在模糊測試時錯誤並非是透過發生例外的方式拋出，這類的程式問題很難被發現。曾經有個工具Go-Sanitizer被提出，並且在Go語言發現這種無例外的問題。然而，在使用工具中有個"Candidate Pick"階段，用戶必須選擇欲插入的斷言並且插入程式碼，但是若用戶不熟悉該產品、Go語言或邏輯，可能會使得插入過程中將語句插入到錯誤的位置或縮排，進而導致非預期的結果，甚至有機會造成程式無法執行。為了解決這個問題，我們提出了一個演算法名叫NEB-San，將找尋和生成斷言和插入到正確位置與縮排全部自動化。在這項研究中，我們將原始碼解析成抽象語法樹(AST)，然後我們根據CWE來定義出匹配標準，用來發現這類問題的準確位置，然後我們再透過抽象語法樹產生斷言節點來反插入到原始抽象語法樹的正確位置並且反轉回程式碼，進而達到所插入的斷言必定在正確的位置以及縮排。我們利用這個想法實現在Python語言中，並提出了一個工具名叫CWE-Sanitizer，這是在Python中的第一個面向錯誤的斷言生成器，他可以全自動生成並且插入斷言到原始碼中來發現這類部拋出例外的錯誤，進而幫助原本傳統的測試技術來透過這些斷言來定位這些錯誤的位置。

Traditionally, we pursue program defects via standard techniques such as unit tests or fuzzing tests. Unit test is talented at discovering predefined defects; fuzzing test is proficient at locating crashing or hanging by executing the program with haphazardly spawned inputs. Nevertheless, some program defects cannot or scarcely be discovered by the classic methods if developers do not think up this scenario, or the execution will not trigger any crash. To find the defects which would not trigger crashes, Go-Sanitizer is proposed to uncover such defects in Golang projects. In the "Candidate Pick" stage of the tool, the user must pick which assertion wants to insert. However, if the user is exotic with this product / Go language or logic, the insertion may be positioned mistakenly or invalid. To unravel this issue, we present an algorithm named NEB-San, endeavoring to generate and insert each assertion statement into the correct place and indentation. In this study, we parse the source code to an abstract syntax tree (AST). Then we demarcate the Matching Criteria based on the CWE to uncover the accurate position of the defect, and then we instrument the code by adding the assertion node to the AST and reverting to code. If we need to check some precondition before the assertion, such as variable type inspection, we wrap the if node beyond the assertion node. We utilize this idea and implement it into Python language. We propose a tool named CWE-Sanitizer, the first bug-oriented assertion generator in Python. It can uncover no-exception defects by automatically generating and inserting the assertion into the original code, allowing traditional testing methods to locate defects by these assertions.