透過Lasso於高維度線性模型之下 訊號的重要性判讀

Abstract

在高維度線性稀疏模型下，Lasso的軌跡解是相當複雜的。採用Lasso的同時，為了選擇微調參數$lambda$，研究Lasso的軌跡解是極為重要的。本文中，當微調參數$lambda$從無限大遞減時，我們給了一個充分條件使得Lasso的軌跡解的支集遞增直到此支集覆蓋住稀疏集(sparsity patten)。根據這個性質，我們可以用變異數分解法來決定微調參數$lambda$要選多少。 Lockhart(2014) 為了決定微調參數$lambda$而提出了 extit{共變異檢定統計} (covariance test statistic)。然而，他們並沒有強調:是否此序貫假設檢定能一路拒絕虛無假設，直到Lasso的解的支集覆蓋住稀疏集(sparsity patten)?為了回答這個問題，我們證明了Lasso的軌跡解有以下的特性，當微調參數$lambda$從無限大遞減時，變數會根據訊號強弱依序進入Lasso的支集。 而我們可以根據進入Lasso的支集的先後來判斷訊號的重要性。此外根據以上的性質，我們可以證明 extit{共變異檢定統計}不會乏適 (underfitting).

In the high dimensional sparse linear regression setting, the Lasso solution path would be rather complicated. And while exploiting Lasso, to choose the tuning parameter $lambda$, analyzing the Lasso solution path is crucial. In this thesis, as the tuning parameter $lambda$ decreases from infinity, we provide a sufficient condition such that the support of the Lasso solution increases until its support recover the sparsity pattern. With this property, exploiting variance decomposition could determine which parameter $lambda$ should we choose. Lockhart2014 proposed the covariance test statistic of Lasso to choose the parameter $lambda$. However, they pay little attention to whether the sequential hypothesis testing could reject the null hypothesis until it recovers the sparsity pattern. In order to deal with this issue, we show that the Lasso solution path would have the ordering property by which we could assess the importance of regressors and the assessment is identical to the oracle importance. And with this property, covariance test statistic would not be underfitting.