Test difficulty, reliability, and discrimination as functions of item difficulty order

Abstract

In an attempt to evaluate the often stated rule that test items should be arranged in an increasing order of difficulty, the effect of item difficulty order on total test reliability, difficulty, and discrimination was investigated in a series of 4 experiments. Each experiment involved a comparison of 2 or more tests, containing the same 40 items and differing only with respect to the order of those items. The differently ordered forms did not, in any of the experiments, differ significantly in test difficulty or test reliability. The results with respect to discrimination were not as clear-cut. However, the results tend to lead to the conclusion that item difficulty order on a power test of facts and principles given in the normal college classroom will not significantly affect these 3 test statistics. (PsycInfo Database Record (c) 2025 APA, all rights reserved)