A Bayesian Analysis of the Information Value of Manipulation and Confounding Checks in Theory Tests

Abstract

Past discussions of manipulation checks and related measures have produced disagreement about their role in tests of theoretical explanations of consumer behavior. We use a Bayesian analysis to examine what such measures contribute to researchers' beliefs about competing theories and suggest when and why manipulation and confounding checks add to the ability to differentiate among alternative theoretical explanations of empirical results. We first focus on the case in which a statistically significant between-group difference on the dependent variable is augmented by information from a single indicator of the intended manipulation and a single indicator of a possible confound. We then examine the implications of multiple indicators and the use of a Bayesian analysis of continuous effect sizes.