On overfitting and post-selection uncertainty assessments

Abstract

In a regression context, when the relevant subset of explanatory variables is uncertain, it is common to use a data-driven model selection procedure. Classical linear model theory, applied naively to the selected submodel, may not be valid because it ignores the selected submodel’s dependence on the data. We provide an explanation of this phenomenon, in terms of overfitting, for a class of model selection criteria.