Safe-Bayesian Generalized Linear Regression

Rianne de Heide, Alisa Kirichenko, Nishant Mehta and Peter Grünwald

What is this paper about?

This paper studies generalized Bayesian inference when the statistical model is misspecified. It uses a learning rate to make Bayesian updating safer under misspecification, proves concentration results for generalized linear models and develops practical algorithms for lasso and logistic regression.

Summary

We study generalized Bayesian inference under misspecification, when the statistical model is “wrong but useful”. Generalized Bayes equips the likelihood with a learning rate. For generalized linear models we show that generalized Bayes with suitable learning rates concentrates around the best approximation to the truth in the model even under severely misspecified noise, provided the true distribution has exponential tails. We derive MCMC samplers for generalized Bayesian lasso and logistic regression and give simulated and real-data examples in which generalized Bayes substantially outperforms standard Bayes.

Topics

Bayesian learning and generalized Bayes

← Research