Carefree multiple testing with e-processes

Yury Tavyrikov, Jelle J. Goeman and Rianne de Heide

What is this paper about?

This paper asks how multiple testing with e-processes should behave when researchers decide adaptively where to collect more data. It argues for using the running supremum of each e-process and shows how adjusters restore FDR control under arbitrary dependence.

Summary

E-processes enable hypothesis testing with ongoing data collection while maintaining Type I error control. However, when testing multiple hypotheses simultaneously, current e-value based multiple testing methods such as e-BH are not invariant to the order in which data are gathered for the different e-processes. This can lead to undesirable situations where a hypothesis rejected at one time is no longer rejected after choosing to gather more data for unrelated e-processes. We argue that multiple testing methods should work with suprema of e-processes. We provide an example showing that e-BH does not control this FDR when applied directly to such suprema, and show that adjusters can be used to ensure FDR-sup control with e-BH under arbitrary dependence.

Topics

E-values and anytime-valid inference · Multiple testing

← Research