Two simple examples for strongunderstandingstrong posterior p-values whose.pdfVIP

Electronic Journal of Statistics Vol. 7 (2013) 2595–2602 ISSN: 1935-7524 DOI: 10.1214/13-EJS854 Two simple examples for understanding posterior p-values whose distributions are far from unform Andrew Gelman Department of Statistics, Columbia University, New York Abstract: Posterior predictive p-values do not in general have uniform dis- tributions under the null hypothesis (except in the special case of ancillary test variables) but instead tend to have distributions more concentrated near 0.5. From dierent perspectives, such nonuniform distributions have been portrayed as desirable (as reflecting an ability of vague prior distri- butions to nonetheless yield accurate posterior predictions) or undesirable (as making it more dicult to reject a false model). We explore this ten- sion through two simple normal-distribution examples. In one example, we argue that the low power of the posterior predictive check is desirable from a statistical perspective; in the other, the posterior predictive check seems inappropriate. Our conclusion is that the relevance of the p-value depends on the applied context, a point which (ironically) can be seen even in these two toy examples. AMS 2000 subject classifications: Primary 62F15, 62C10, 62F03. Keywords and phrases: Bayesian inference, model checking, posterior predictive check, p-value, u-value. Received February 2013. 1. Introduction Bayesian predictive checking generalizes classical hypothesis testing by averag- ing over the posterior distribution of the unknown parameter vectors rather than fixing them at some point estimate (Rubin, 19