Qualitative Robustness of Tests

Abstract

A test is qualitatively robust by definition if its sequence of – n -1 log transformed P values, n being a measure of the sample size, is continuous as a point function of the observations and weakly equicontinuous as a function of discrete probability measures. This definition is applitable both to unconditional and to conditional tests. Under weak regularity conditions, an unconditional test is qualitatively robust if and only if its test statistic is continuous; a counterexample shows that conditional tests do not share this property. The sample mean, Student's t and [Ybar] — [Xbar] permutation tests are not qualitatively robust; the sign, Wilcoxon, Huber censored likelihood, and normal scores tests are qualitatively robust.