Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses

The Annals of Statistics · 2006 · Volume 34 · Issue 1

Abstract

We consider the problem of estimating the number of false null hypotheses among a very large number of independently tested hypotheses, focusing on the situation in which the proportion of false null hypotheses is very small. We propose a family of methods for establishing lower 100(1−α)% confidence bounds for this proportion, based on the empirical distribution of the p-values of the tests. Methods in this family are then compared in terms of ability to consistently estimate the proportion by letting α→0 as the number of hypothesis tests increases and the proportion decreases. This work is motivated by a signal detection problem that occurs in astronomy.

Keywords

62H15, 62J15, 62P35, Hypothesis testing, Multiple comparisons, Sparsity

Enable JavaScript for citation tools and additional paper interactions.