Is There a Citation Bias in the Judgment and Decision Literature?

Abstract

Christensen-Szalanski and Beach (1984) reported that studies showing "poor" decision making performance are cited more frequently than studies showing "good" performance and concluded that a citation bias exists in the judgment and decision making literature. The present analysis was conducted to examine the empirical and conceptual basis of this disparity. A taxonomy of judgment and decision making studies was constructed to facilitate a more differentiated comparison than the dichotomous classifications used in the Christensen-Szalanski and Beach analysis. The 84 articles used by Christensen-Szalanski and Beach were classified into this new performance outcome taxonomy; sample size, level of realism in the research context, journal type, and journal impact were also examined. The results show that the empirical basis for the citation-bias hypothesis has yet to be demonstrated conclusively because the citation disparity observed by Christensen-Szalanski and Bean is weak in magnitude and is possibly unreliable. Furthermore, the two sets of articles tend to be published in different types of journals and tend to report qualitatively different kinds of evidence, raising the possibility that any observed citation-rate disparity could stem from these differences rather than from a citation bias. Finally, some of the assumptions underlying the citation-bias hypothesis are challenged and alternative hypotheses regarding the citation-rate disparity are discussed.