Hindsight bias in an application of the Angoff method for setting cutoff scores

Abstract

This study used the hindsight bias literature to examine the effects of providing test item answers on the level and predictive accuracy of Angoff cutoff score judgments. Subjects, randomly assigned to either an answers or a no-answers group, made Angoff judgments on items testing knowledge of driving law and vehicle operation. Items were categorized into subtests based on empirical difficulty. Results indicated that providing answers interacted with subtest difficulty to influence the level of Angoff judgments. On the easiest subtest, those in the answers group made higher judgments than did those in the no-answers group; the reverse effect was observed on the most difficult subtest. Subjects in the answers group more accurately predicted empirical item difficulties than did subjects in the no-answers group. (PsycInfo Database Record (c) 2025 APA, all rights reserved)