數值狀態覆蓋率驅動式模糊測試之評估

Abstract

程式碼覆蓋率是模糊測試技術中非常重要且有效的指標，但在驅動模糊測試工具發現漏洞方面，仍存在根本性的限制：其無法有效引導工具探索變數的數值，導致即使具漏洞的程式碼已被多次執行，仍可能因未能提供觸發漏洞所需的特定變數數值而錯失漏洞。 YFuzz 引入「數值狀態覆蓋率」（value state coverage）來因應此問題，藉由監控與安全相關的變數之數值，引導模糊測試工具滿足觸發漏洞所需的數值條件。然而，儘管其概念具說服力，且過去評估中的成果看似顯著，我們的全面性分析揭露 YFuzz 存在重大的實作錯誤與數項評估問題，這些問題削弱了 YFuzz 原始論述中聲稱其效果的有效性。其中最關鍵的發現是一項實作錯誤，使高達 83.44% 的安全變數並非理論中應被監控的變數，導致 YFuzz 的理論方法與實際實作之間出現落差。 為釐清這些問題並深入了解 YFuzz 的實際效果，我們修正了實作錯誤，並使用 15 個程式對其進行全面的性能評估。實驗結果顯示，YFuzz 的數值狀態覆蓋率相較於 AFL++ 提升了 31.62%，但伴隨著 18.30% 的執行速度損耗，以及 29.02% 的程式碼覆蓋率下降。 透過消融實驗，我們發現藉由參數最佳化可使安全相關變數的覆蓋率提升達 93%，並有助於增強漏洞挖掘能力。然而，與 AFL++ 和 DDFuzz 的比較顯示，儘管 YFuzz 在技術指標上表現優異，卻未能發現任何獨特的漏洞，說明數值狀態覆蓋率的提升未必能直接轉化為更佳的漏洞挖掘效果。 我們的研究提供了對 YFuzz 能力更深入的理解，展示數值狀態覆蓋率在目前設計下的限制。我們也討論了改進 YFuzz 與數值狀態覆蓋率方法的可能方向，例如導入自適應引導策略，並分析評估新的模糊測試技術時所面臨的挑戰。這些觀察對以數值狀態為基礎的模糊測試方法的後續研究提供了建議與參考方向。

Despite being one of the most common and effective methods in fuzzing, code coverage-guided fuzzing has limitations in detecting vulnerabilities that require specific variable values to trigger, even when the vulnerable code paths are already covered. YFuzz addresses this issue by introducing value state coverage, which monitors the values of security-sensitive variables to guide fuzzers toward satisfying the data requirements necessary for triggering vulnerabilities. However, despite its appealing concept and previously reported success, our analysis reveals a critical implementation flaw and several evaluation issues that threaten the validity of YFuzz's original claims. Most significantly, we discover that 83.44% of monitored security-sensitive variables are incorrectly identified, creating a fundamental disconnect between YFuzz's theoretical design and its actual implementation. To address these issues and rigorously evaluate YFuzz, we correct the implementation problem and conduct a comprehensive re-evaluation on 15 programs—more than the original study and typical fuzzing research. Our results show that YFuzz achieves a 31.62% improvement in value state coverage over AFL++ but incurs 18.30% runtime overhead and 29.02% reduction in code coverage. Through ablation studies, we demonstrate a 93% improvement in security-sensitive variable coverage through parameter optimization, which also enhances its bug discovery capability. However, comparison against AFL++ and DDFuzz shows that YFuzz does not discover any unique vulnerabilities despite these technical improvements. This suggests that gains in value state coverage do not necessarily translate into better bug discovery capability. Our findings offer a clearer understanding of YFuzz's strengths and limitations, and highlight the current constraints of value-state coverage in its existing design. We also discuss directions for improving both YFuzz and value state based approaches—such as incorporating adaptive guidance strategies—and examine the challenges of evaluating novel fuzzing techniques like YFuzz. These insights provide guidance for the future development of fuzzing methods grounded in value state coverage.