基於因素分析與變數相對重要性之變數選擇方法及其應用

Abstract

高維度小樣本資料是資料探勘領域中的一個挑戰，這種類型的資料在生物資訊領域中尤為常見，比如說基因表現微陣列資料記載了人體成千上萬個基因，而樣本數卻只有幾百。要如何從眾多的基因中找出關鍵基因以預測產生疾病的可能性是一大課題。以統計領域中迴歸分析方法的觀點來看，這是變數選擇的問題，至今有諸多學者提出不同方法，而其中的癥結點是該如何處理變數的「共線性」問題以及變數的「組合」解釋力。 針對共線性問題，J.W. Johnson (2000) [16] 利用 R.M. Johnson (1966) [17] 所提出的「最佳近似正交轉換方法」轉換自變數 (X) 得到正交的中介變數 (Z)，再透過中介變數 Z 計算變數相對權重 (Relative Weight)，然而該方法僅適用於樣本數 (n) 大於變數數量 (p) 且變數間無線性相關，也就是 X 為滿秩 (full-rank) 之情況，故 Shen and Chen (2020) [25] 延伸 J.W. Johnson 的相對權重 (Relative Weight)，提出「普適型相對重要性」 (Comprehensive Relative Importance) 計算方法，使得任何 n 及 p 的數值型資料皆可適用，然而該方法在 p 非常大且 p>>n 時，所需之矩陣計算量也隨之過大。 本研究的目的為建立在以上學者針對相對重要性議題所提出之方法上，針對 p 非常大的問題，提供一個在因素分析的框架上計算變數相對重要性的方法。假設在高維度問題中自變數量 p 很大，但 X 矩陣的 rank 只有 r，本研究首先利用因素分析中的因素萃取及因素變異最大化 (Varimax) 旋轉獲得的 r 個因素，取代原有相對重要性計算中的「最佳近似正交轉換」的 p 個中介變數，再計算得到變數相對重要性後排序選取重要變數進入迴歸模型。本研究同時也嘗試創新因素萃取方法，增加考量對應變數的解釋力，也就是計算自變數與應變數之間的相關係數，將其排序並選擇前 r (r<<p) 個大的變數，接著透過最佳近似正交轉換得到 r 個垂直中介變數 Z，再對 r 個 Z 進行 Varimax 旋轉以得到「r 個因素」，之後再計算得到變數相對重要性。 原本 Shen and Chen (2020) 都是在 n 維空間的 p 維子空間操作，但本研究方法降到 r 維空間計算變數相對重要性，由此帶來的好處是計算量下降。另外，Shen and Chen (2020) 並未真正解決共線性問題，也就是自變數在計算上是透過不垂直的中介變數解釋應變數的變異，但本研究方法確保中介變數互相垂直，因此得以合理使用中介變數分配相對重要性到原始自變數上，由此亦更具解釋性。 本文使用模擬案例以及基因表現資料進行方法驗證。從模擬案例中可以發現本研究方法可以找出實驗設定的垂直因素，在高度共線性的情形，其變數選擇表現及計算效率也比 Shen and Chen (2020) 變數相對重要性方法更好，達到利用因素分析計算變數相對重要性的目標。在基因表現資料的建模預測中，本研究方法的預測結果大部分較傳統的變數選擇方法好，且能夠在某些案例中與 Shen and Chen (2020) 的方法表現相當。

High-dimensional data with a small sample size is a challenge for researchers in data analysis. This is especially true in the field of bio-informatics, where datasets, such as gene expression microarray data, consist of tens of thousands of features but only have a sample size of hundreds. Hence, the question arises: how can we find the key features among those genes for modeling and prediction? From the perspective of Regression Analysis in the Statistics domain, this is a well-known “variable selection” problem. However, no consensus has been reached regarding the treatment of the “multicollinearity” that often arises in high-dimensional settings. Regarding the problem of multicollinearity, J.W. Johnson (2000) [16] used the “Best-Fitting Orthogonal Approximation of X” method proposed by R.M. Johnson (1966) [17] to transform the independent variables (X) into their orthogonal counterparts (Z) and calculate the “Relative Weight” of each feature. However, this method is limited to cases where the sample size (n) is greater than the number of features (p) and when the design matrix X is full-rank. Shen and Chen (2020) [25] extended the “Relative Weight” method proposed by J.W. Johnson (2000) and introduced the method “Comprehensive Relative Importance (CRI).” This method can be applied to any design matrix, irrespective of singularity or size. Nevertheless, the computational complexity increases as p is very large and p>>n. The purpose of this study is to propose a new framework inspired by Factor Analysis to address the computational complexity problem in Shen and Chen (2020), specifically in the high-dimensional setting. Assume that p is very large, but the rank of X is only r. Our study first adopts the Factor Extraction and Varimax Factor Rotation process in Factor Analysis in order to find r factors, which serve as the orthogonal basis that spans the subspace of the design matrix. We replace the original p intermediate variable Zs, which are the Best-Fitting Orthogonal Approximation of X, with these r factors for calculating the relative importance. We then select the top-ranked variables by their relative importance into the regression analysis model. This study also attempts to innovate the Factor Extraction method, in which we consider the effect of predictors on the dependent variable. We calculate the Pearson simple correlation between the predictors and the dependent variable, sort the values in descending order and select the first r large variables. We then transform the factors to the r orthogonal intermediate variables Z using the Best-Fitting Orthogonal Approximation of X proposed by R.M. Johnson (1996). Finally, we perform Varimax Rotation on Z and calculate the relative importance. While Shen and Chen (2020) perform all the calculation in the p-dimensional subspace contained in the n-dimensional space, this study reduces to r-dimensional subspace to calculate relative importance. This reduction results in a reduction in computational complexity. Furthermore, Shen and Chen (2020) use non-orthogonal intermediate variables to explain the variation in the dependent variable, but this study ensures that orthogonal intermediate variables are used in the calculation. Hence, our proposed method has higher interpretability. Our proposed method is validated using simulation cases and gene expression microarray datasets. From the results of the simulation cases, we can see that our method can indeed identify the corresponding factors that are designed in the simulations to influence both the dependent and independent variables. Moreover, in cases where predictors are highly correlated, our proposed method can select the truly important variables better and requires less computation compared to the method by Shen and Chen (2020). Finally, when applied to gene expression datasets for disease prediction, the proposed method is comparable to the method of Shen and Chen (2020) and outperforms most other popular feature selection algorithms in terms of classification performance in most cases.