Regularization and the small-ball method I: Sparse recovery
Guillaume Lecué, Shahar Mendelson
The Annals of Statistics · 2018 · Volume 46 · Issue 2
Abstract
We obtain bounds on estimation error rates for regularization procedures of the form \begin{equation*}\hat{f}\in\mathop{\operatorname{argmin}}_{f\in F}(\frac{1}{N}\sum_{i=1}^{N}(Y_{i}-f(X_{i}))^{2}+\lambda \Psi(f))\end{equation*} when $\Psi$ is a norm and $F$ is convex. Our approach gives a common framework that may be used in the analysis of learning problems and regularization problems alike. In particular, it sheds some light on the role various notions of sparsity have in regularization and on their connection with the size of subdifferentials of $\Psi$ in a neighborhood of the true minimizer. As “proof of concept” we extend the known estimates for the LASSO, SLOPE and trace norm regularization.
Keywords
60K35, 62C20, 62G05, 62G08, 62G20, Empirical processes, High-dimensional statistics
Enable JavaScript for citation tools and additional paper interactions.