Addressing the diversity-validity dilemma in personnel selection: Unraveling the impact of multipenalty optimized regression in varied testing scenarios

Abstract

Researchers and practitioners have long grappled with balancing the goals of selecting a high-performing and diverse workforce. Recently, Rottman et al. (2023) proposed a new approach to address these goals, which we refer to as multipenalty optimized regression (MOR). MOR extends ridge regression by adding a penalty term that minimizes group differences when fitting the model. Although MOR has shown potential, there are unknowns, including whether MOR is consistently effective in typical selection settings, what conditions impact MOR effectiveness, and whether MOR performs similarly to other multiobjective optimization methods, such as Pareto-normal boundary intersection (Pareto-NBI). Using Monte Carlo simulations (Study 1), we investigated MOR effectiveness and compared it with traditional scoring methods (ridge regression, ordinary least squares, unit weighting) and Pareto-NBI across several factors: (a) number of scales (and corresponding items), (b) operationalization (item or scale), (c) magnitude of predictor criterion-related validity, (d) magnitude of predictor subgroup differences, (e) calibration sample size, and (f) proportion of minorities in the calibration sample. Compared with traditional methods, MOR frequently produced solutions with comparable criterion-related validity but with consistently less adverse impact risk. Pareto-NBI and MOR were similarly effective in performing dual optimization, though MOR was more effective at very small sample sizes (e.g., N < 150) with item-level scoring. Pareto-NBI also became computationally intensive with many predictors, making MOR better suited for big data. Finally, in Study 2, MOR exhibited similar criterion-related validity and lower adverse impact risk relative to other methods across six real-life assessment contexts. We provide recommendations for using multiobjective optimization methods in personnel selection. (PsycInfo Database Record (c) 2025 APA, all rights reserved)