Deflated HeteroPCA: Overcoming the curse of ill-conditioning in heteroskedastic PCA
The Annals of Statistics · 2025 · Volume 53 · Issue 1
Abstract
This paper is concerned with estimating the column subspace of a low-rank matrix X⋆∈Rn1×n2 from contaminated data. How to obtain optimal statistical accuracy while accommodating the widest range of signal-to-noise ratios (SNRs) becomes particularly challenging in the presence of heteroskedastic noise and unbalanced dimensionality (i.e., n2≫n1). While the state-of-the-art algorithm HeteroPCA emerges as a powerful solution for solving this problem, it suffers from “the curse of ill-conditioning,” namely, its performance degrades as the condition number of X⋆ grows. In order to overcome this critical issue without compromising the range of allowable SNRs, we propose a novel algorithm, called Deflated−HeteroPCA, that achieves near-optimal and condition-number-free theoretical guarantees in terms of both ℓ2 and ℓ2,∞ statistical accuracy. The proposed algorithm divides the spectrum of X⋆ into well-conditioned and mutually well-separated subblocks, and applies HeteroPCA to conquer each subblock successively. Further, an application of our algorithm and theory to two canonical examples—the factor model and tensor PCA—leads to remarkable improvement for each application.
Keywords
62F10, 62H25, factor models, heteroskedastic noise, Principal component analysis (PCA), Tensor PCA, the curse of ill-conditioning
Enable JavaScript for citation tools and additional paper interactions.