Retest-reliability by a movie technique of test administrator's judgments of performance in progress

Abstract

A motion picture technique for recording a work sample performance test (on a drill point grinding task) is described. Five examiners each viewed the film and scored the test twice. Intra-observer consistency (proportion of items answered alike twice) ranged from 64.3% to 100%, with a mean of 82.8%. "The mean intra-observer consistency… was adequate. However, the range… was great enough to warrant… careful investigation of intra-examiner reliability prior to the administration of work sample performance tests." (PsycInfo Database Record (c) 2025 APA, all rights reserved)