Thep-filter: multilayer false discovery rate control for grouped hypotheses

Journal of the Royal Statistical Society: Series B (Statistical Methodology) · 2017 · Volume 79 · Issue 4

Abstract

In many practical applications of multiple testing, there are natural ways to partition the hypotheses into groups by using the structural, spatial or temporal relatedness of the hypotheses, and this prior knowledge is not used in the classical Benjamini–Hochberg procedure for controlling the false discovery rate (FDR). When one can define (possibly several) such partitions, it may be desirable to control thegroup FDRsimultaneously for all partitions (as special cases, the ‘finest’ partition divides thenhypotheses intongroups of one hypothesis each, and this corresponds to controlling the usual notion of FDR, whereas the ‘coarsest’ partition puts allnhypotheses into a single group, and this corresponds to testing the global null hypothesis). We introduce thep-filter, which takes as input a list ofn p-values andM⩾1 partitions of hypotheses, and produces as output a list ofnor fewer discoveries such that the group FDR is provablysimultaneouslycontrolled for all partitions. Importantly, since the partitions are arbitrary, our procedure can also handle multiple partitions which are non-hierarchical. Thep-filter generalizes two classical procedures—whenM=1, choosing the finest partition intonsingletons, we exactly recover the Benjamini–Hochberg procedure, whereas, choosing instead the coarsest partition with a single group of sizen, we exactly recover the Simes test for the global null hypothesis. We verify our findings with simulations that show how this technique can not only lead to the aforementioned multilayer FDR control but also lead to improvedprecisionof rejected hypotheses. We present some illustrative results from an application to a neuroscience problem with functional magnetic resonance imaging data, where hypotheses are explicitly grouped according to predefined regions of interest in the brain, thus allowing the scientist to employ field-specific prior knowledge explicitly and flexibly.

Keywords

False discovery rate, Grouped hypotheses, Multilayer, Multilevel, Multiple testing, Multiresolution, p-filter

Enable JavaScript for citation tools and additional paper interactions.