Effects of rater training: Creating new response sets and decreasing accuracy

Abstract

Compared the extent of leniency error, the extent of halo effect, and the accuracy of ratings on behavioral expectation scales across 72 undergraduates in 3 groups of raters. Group 1 raters underwent a training program on rating error, involving definitions, graphic illustrations, and numerical examples of leniency and halo. Group 2 heard lectures on the multidimensionality of teacher performance, generated and defined dimensions of performance, discussed behavioral examples of each dimension, and attempted to develop stereotypes of effective and ineffective performance. Group 3 received no training. Following the training, 2 hypothetical ratees were rated on 13 dimensions of performance. Ratings from Group 1 had significantly less leniency and halo error than ratings from Groups 2 and 3. When ratings were compared to previously developed true scores on the ratees, significantly less accuracy was found for Group 1 than for the other 2 groups. No significant differences were found between the Group 2 and Group 3 ratings. (15 ref) (PsycInfo Database Record (c) 2025 APA, all rights reserved)