Reliability of Peace Corps Selection Boards: A study of interjudge agreement before and after board discussions

Abstract

The most crucial link in the Peace Corps selection process is the Advisory Selection Board, where a comprehensive pool of assessment data on each trainee is evaluated and discussed. In an effort to better understand this important process of collective clinical judgment, 9 Peace Corps Selection Boards were studied. Agreement between Board participants on the overall suitability of each Peace Corps trainee prior to Board discussion was compared with that reached after Board discussion. In general, the findings from 9 Selection Boards appeared remarkably similar, indicating that Board discussions tend to (1) decrease the suitability ratings for the average trainee, (2) increase the average dispersion of ratings for the group of trainees, and (3) increase the degree of consensus among Board participants. The unusually high consensus among participants after Board discussions attests to the rationality—though not necessarily the validity—of the Peace Corps selection process. (PsycInfo Database Record (c) 2025 APA, all rights reserved)