A Goodness-of-Fit Assessment for General Learning Procedures in High Dimensions

Journal of the American Statistical Association · 2026 · Volume 121 · Issue 553

DOI: 10.1080/01621459.2025.2529602

Abstract

Black-box learners have demonstrated remarkable success across various fields due to their high predictive accuracy. However, the complexity of their learning procedures poses significant challenges in evaluating whether a given learner has achieved optimal performance on datasets with unknown data-generating mechanisms. We propose a general goodness-of-fit test for assessing different learning procedures involving high-dimensional predictors, encompassing methods from classical linear regression to advanced neural networks. Our goodness-of-fit test leverages data-splitting, using the test set to evaluate the black-box learner trained on the training set. By examining the cumulative covariance of the residuals, our method can effectively handle high-dimensional predictors. Extensive simulations and three real data analyses validate the effectiveness of our method. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Keywords

Cumulative covariance, General learning procedures, High-dimensional predictors, Model checking

Enable JavaScript for citation tools and additional paper interactions.