Selecting Penalty Parameters of High-Dimensional M-Estimators Using Bootstrapping after Cross Validation

Abstract

We develop a new method for selecting the penalty parameter for ℓ 1 -penalized M-estimators in high dimensions, which we refer to as bootstrapping after cross validation. We derive rates of convergence for the corresponding ℓ 1 -penalized M-estimator and also for the post-ℓ 1 -penalized M-estimator, which refits the nonzero entries of the former estimator without penalty in the criterion function. We demonstrate via simulations that our methods are not dominated by cross validation in terms of estimation errors and can outperform cross validation in terms of inference. As an empirical illustration, we revisit Fryer ( 2019 ), who investigated racial differences in police use of force, and confirm his findings.