The Effect of Simulation Order on Level Accuracy and Power of Monte Carlo Tests

Journal of the Royal Statistical Society: Series B (Statistical Methodology) · 1989 · Volume 51 · Issue 3

DOI: 10.1111/j.2517-6161.1989.tb01440.x

Abstract

We discuss the effect of simulation order on level accuracy and power of Monte Carlo tests, in a very general setting. Both parametric problems, with or without nuisance parameters, and nonparametric problems are treated by a single unifying argument. It is shown that if the level of a Monte Carlo test is known only nominally, not precisely, then the level error of a Monte Carlo test is an order of magnitude less than that of the corresponding asymptotic test. This result is available whenever the test statistic is asymptotically pivotal, even if the number of simulations is held fixed as the sample sizenincreases. It implies that Monte Carlo methods are a real alternative to asymptotic methods. We also show that, even if the number of simulations is held fixed, a Monte Carlo test is able to distinguish between the null hypothesis and alternative hypotheses distantn–1/2from the null.

Keywords

bootstrap, hypothesis test, level error, monte carlo test, nuisance parameter, power

Enable JavaScript for citation tools and additional paper interactions.