Ceilings in the Reliability and Validity of Performance Ratings: The Case of Expert Raters

Abstract

Ratings gathered from expert raters under nearly ideal conditions (world class judges of the figure-skating competition at the 1984 Olympics) were examined for evidence of reliability and validity. In general, the results suggest that the ceiling on rating reliability and validity may be higher than indicated by previous research.