Variable Selection in Linear Regression with Group Structure via the Group Lasso and Mallows' Cp

Abstract

We consider the problem of selecting grouped variable in linear regression via the group Lasso and Mallows' Cp, especially when the columns in the full design matrix are orthogonal. We address two questions. Since Mallows' Cp is derived to be prediction optimal, how well the group Lasso coupled with Cp-criterion performs on selecting or dropping grouped variables? Since the group Lasso exploits additional group structure, will it perform better than Lasso on selecting the correct model? We propose that the behavior of the group Lasso coupled with Cp-criterion on selecting or dropping a grouped variable is like the detection of the grouped variable coming from χ2p or χ'2p. Moreover, we observe that the group Lasso coupled with Cp-criterion leads to a over-fitted regression model. The group structures do not always encourage us to select a better model when we compare that with Cp-Lasso.