Robust and consistent model evaluation criteria in high-dimensional regression

Abstract: In the last two decades, sparse regularization methods such as the LASSO have been applied in various fields. Most of the regularization methods have one or more regularization parameters, and to select the value of the regularization parameter is essentially equal to select a model, thus we need to determine the regularization parameter adequately. Regarding the determination of the regularization parameter in the linear regression model, we often apply the information criteria like the AIC and BIC, however, it has been pointed out that these criteria are sensitive to outliers and tend not to perform well in high-dimensional settings. Outliers generally have a negative influence on not only estimation but also model selection, consequently, it is important to employ a selection method that is robust against outliers. In addition, when the number of explanatory variables is quite large, most conventional criteria are prone to select unnecessary explanatory variables. In this paper, we propose model evaluation criteria via the statistical divergence with excellence in robustness in both of parametric estimation and model selection. Furthermore, our proposed criteria simultaneously achieve the selection consistency with the robustness even in high-dimensional settings. We also report the results of some numerical examples to verify that the proposed criteria perform robust and consistent variable selection compared with the conventional selection methods.