Ranking and selection in large-scale inference of heteroscedastic units
Bowen Gang, Luella Fu, Gareth M. James, Wenguang Sun
The Annals of Applied Statistics · 2026 · Volume 20 · Issue 1
Abstract
The allocation of limited resources to a large number of potential candidates presents a pervasive challenge. In the context of ranking and selecting top candidates from heteroscedastic units, conventional methods often result in overrepresentations of subpopulations, and this issue is further exacerbated in large-scale settings where thousands of candidates are considered simultaneously. In particular, we consider this problem in ranking schools based on socioeconomic performance gaps in standardized testing. To address this challenge, we propose a new multiple comparison framework that incorporates a modified power notion to prioritize the selection of important effects and employs a novel ranking metric to assess the relative importance of units. We develop both oracle and data-driven algorithms and demonstrate their effectiveness in controlling the error rates and achieving optimality. We evaluate the numerical performance of our proposed method using simulated and real data. The results show that our framework enables a more balanced selection of effects that are both statistically significant and practically important and results in an objective and relevant ranking scheme that is well-suited to practical scenarios.
Keywords
composite null hypotheses; Compound decision theory; deconvolution estimates; Empirical Bayes; False discovery rate; weighted multiple testing;
Enable JavaScript for citation tools and additional paper interactions.