Supervised homogeneity fusion: A combinatorial approach

Wen Wang, Shihao Wu, Ziwei Zhu, Ling Zhou, Peter X.-K. Song

The Annals of Statistics · 2024 · Volume 52 · Issue 1

Abstract

Fusing regression coefficients into homogeneous groups can unveil those coefficients that share a common value within each group. Such groupwise homogeneity reduces the intrinsic dimension of the parameter space and unleashes sharper statistical accuracy. We propose and investigate a new combinatorial grouping approach called L0-Fusion that is amenable to mixed integer optimization (MIO). On the statistical aspect, we identify a fundamental quantity called MSE grouping sensitivity that underpins the difficulty of recovering the true groups. We show that L0-Fusion achieves grouping consistency under the weakest possible requirement of the grouping sensitivity: if this requirement is violated, then the minimax risk of group misspecification will fail to converge to zero. Moreover, we show that in the high-dimensional regime, one can apply L0-Fusion with a sure screening set of features without any essential loss of statistical efficiency, while reducing the computational cost substantially. On the algorithmic aspect, we provide an MIO formulation for L0-Fusion along with a warm start strategy. Simulation and real data analysis demonstrate that L0-Fusion exhibits superiority over its competitors in terms of grouping accuracy.

Keywords

60J05, 62P10, 90C11, clustering, Dimension reduction, linear model, mixed integer optimization, supervised learning

Enable JavaScript for citation tools and additional paper interactions.