Optional Stopping with Bayes Factors: A Categorization and Extension of Folklore Results, with an Application to Invariant Situations

Allard Hendriksen, Rianne de Heide and Peter Grünwald

What is this paper about?

This paper disentangles several different claims that are often all described as “Bayesian methods allow optional stopping”. It distinguishes stopping-rule independence, posterior calibration and frequentist robustness, proves general results for these notions, and studies the role of group invariance and improper priors.

Summary

It is often claimed that Bayesian methods, in particular Bayes-factor methods for hypothesis testing, can deal with optional stopping. We distinguish three mathematical meanings of this claim: stopping-rule independence, posterior calibration and (semi-)frequentist robustness to optional stopping. We prove general measure-theoretic results for these notions, including new results for calibration and frequentist robustness. By allowing non-integrable measures based on improper priors, we obtain strong results for models with nuisance parameters satisfying a group invariance such as location or scale. We also discuss the practical relevance of these guarantees and conclude that actual performance under optional stopping depends crucially on the models, priors and goal of the analysis.

Topics

Bayesian learning and generalized Bayes · E-values and anytime-valid inference · Foundations of statistics, probability and learning

← Research