基於實驗研究機器學習於因果發現的作用與影響

Abstract

因果推論長久以來一直是人們關注的核心問題，其中又以確認事物的因果關係最為受到重視。檢驗因果關係通常以實驗作為標準，然而有時囿於實驗倫理、實驗方法或資金等限制，這類實驗往往難以執行，這促使人們發展從觀測資料中直接揭示因果結構的方法。不過因果發現本質上並不是清楚建構的問題，因為可能有多個因果模型同時生成相同的觀測分布，因此函數型因果發現方法引入了結構性假設，通過限制因果機制所遵循的函數類別來約束模型空間。這類基於函數假設進行推論的方法與機器學習的領域高度重合。

在本研究中，我們探討了機器學習在函數型因果發現中的作用，並著重關注樣本複雜度與模型不匹配等問題。我們量化了雙變量函數型因果發現中，各個環節對樣本數量的需求，同時指出回歸為高樣本複雜度的主因。此外我們發現，回歸與統計獨立性檢定的資料重疊與否，將對因果發現的正確率產生影響。據此我們提出了如何在雙變量函數型因果發現問題使用資料的策略框架。我們建議在各個因果發現的環節重複使用資料，以達成最佳正確率。此外我們觀察到過少參數的回歸模型無法達成因果發現的任務。

在當今的時代，許多企業與研究機構會釋出大量訓練好的模型供公眾使用。透過利用這些模型，我們或能夠進一步降低因果發現對樣本數量的要求。經過研究，我們提出了利用這些預訓練模型，以輔助三變量因果發現的新方法。在這篇研究中，我們聚焦於線性模型。利用線性的預訓練模型，以還原出現性結構因果模型的係數，是本方法的主要概念。在極少量樣本的情況下，如少於100樣本，我們的方法達到了接近90%的正確率，遠勝於傳統方法。

Causal inference has long been a central problem of human interest. While experiments have traditionally been the gold standard for identifying causal relations, such methods are often infeasible due to ethical, practical, or financial constraints. This drives the need to develop methods for uncovering causal structure directly from observational data. Notice that causal discovery is an ill-posed problem, as multiple causal models may be consistent with the same observed distribution. To address this, functional causal discovery introduces structural assumptions by restricting the class of functions governing the causal mechanisms. This reformulates the task as identifying functional dependencies among variables, relying heavily on regression and statistical independence tests. These methods lie in the domain of machine learning.

In this work, we study the role of machine learning in functional causal discovery, with particular focus on issues such as sample complexity and model mismatch. Through empirical analysis, we quantify the sample demands of each component in bivariate functional causal discovery and show regression as the primary bottleneck. In addition, we discovered that whether data overlap between the regression and the statistical independence test makes an impact on causal discovery accuracy. And propose a framework on the strategy of using data in bivariate functional causal discovery. We recommend reusing data throughout the functional causal discovery pipeline to achieve the best accuracy. In addition, we observe that the underparametrization may harm causal discovery.

Nowadays, there are plenty of well-trained models released to the public. If we could utilize these models, it would provide the opportunity to reduce the sample complexity. We propose a novel method that leverages these well-pretrained models to assist 3-variable causal discovery. We mainly focus on the linear cases. The main idea builds on recovering the coefficients in linear structural causal models with the linear pretrained models. Our approach achieves nearly 90\% accuracy in fairly small sample size regimes (e.g., less than 100), which is far better than the conventional methods.