Transfer Learning for Covariance Matrix Estimation: Optimality and Adaptivity

T. Tony Cai, Dongwoo Kim, Yicheng Li, Dongdong Xiang

Journal of the American Statistical Association · Fri

DOI: 10.1080/01621459.2026.2725280

Abstract

Transfer learning, which leverages auxiliary data to improve inference in a target domain, has become a powerful tool in machine learning. In this work, we study minimax and adaptive estimation of large bandable covariance matrices under a transfer learning framework. We first establish the minimax rate of convergence under squared spectral norm loss and propose a rate-optimal estimator. Our analysis reveals phase transitions determined by the source-target disparity, the source sample size, the dimension, and the two smoothness indices. We then develop an adaptive estimator, a fully data-driven procedure that does not require the unknown smoothness and disparity parameters. In most regimes, it directly attains the minimax optimal rate without additional cost of adaptation. Simulation studies support our theoretical results and demonstrate the practical effectiveness of the proposed adaptive procedure. We further illustrate its utility in a phoneme classification task, where incorporating related phonemes as auxiliary data significantly improves classification performance.

Keywords

Adaptive covariance estimation, Covariance matrix, High-dimensional statistics, Minimax rate of convergence, Transfer learning

Enable JavaScript for citation tools and additional paper interactions.