Testing the constancy of validity with computer-generated sampling distributions of the multiplicative model variance estimate: Results for petroleum industry validation research

Abstract

A comparison of computer-generated sampling distributions showed that increases in total N resulting from greater sample size/validity study reduced the sampling variation of multiplicative model estimates of true validity variance more than did increases in total N resulting from a larger number of studies. Validity coefficients, range restriction, and criterion reliability data for 5 tests predicting job performance and training performance were obtained from studies conducted at petroleum industry plants. Those tests included the Richardson-Bellows-Henry Ability, Arithmetic Reasoning, and Chemical Comprehension Tests; the Bennett Mechanical Comprehension Tests; and the Mathematics scale of the California Achievement Tests. Studies for 3 job categories (operation, maintenance, and laboratory) were combined for analysis to maximize sensitivity to job and plant differences. Sampling distributions of true validity variance estimates were generated for each test–criterion type combination. A comparison of actual variance estimates with critical values indicated that 3 of the 5 tests did not vary significantly in true validities for job performance or for training criteria. Results for 2 arithmetic reasoning tests support the existence of variation in true validities for job performance and for training criteria. (19 ref) (PsycInfo Database Record (c) 2025 APA, all rights reserved)