Online inference under over-parameterized models with hidden confounders

Shuyan Chen, Xingdong Feng, Yeheng Ge, Tao Li, Mengyun Wu

Journal of the Royal Statistical Society: Series B (Statistical Methodology) · Mon

Abstract

In this paper, we study online estimation and inference of regression coefficients in the presence of hidden confounders by leveraging over-parameterized models. Unlike existing offline approaches that rely on factor and sparse models, our closed-form estimator simultaneously removes hidden-confounder bias and is directly applicable to streaming data. Using tools from random matrix theory, we analyse phase transition phenomena in the variance of the coefficient estimator that arise as the sample size transitions from being smaller than to larger than the number of predictors. Notably, we show that adding more covariates only slightly affects the estimator’s variance, mitigating concerns about variance inflation in over-parameterized settings. We validate the effectiveness of our method for both individual coefficient inference and multiple testing through simulations and applications to two real datasets.

Keywords

dense models, high-dimensional data, nonsparsity, random matrix theory, ridge regression

Enable JavaScript for citation tools and additional paper interactions.