Latest Comments
« San Francisco Sourdough Bread | Main | The Ph.D. Grind »

Be Credible by being Bayesian

Recently at work we encountered a problem with the "standard" way of computing confidence intervals for a Binomial proportion using the Normal approximation. That is,
At small sample sizes and high sample proportions, this is too conservative. For example, with a sample size of 10, the confidence intervals for a sample proportion of 0.8 at 95% confidence level, the confidence interval based on the Normal approximation is [0.55, 1].

We can easily see that this confidence interval is too wide as it includes p = 1.0, but we already saw 2 negative examples, which flies in the face of the evidence that p = 1.0 should be on the table.

There are many alternative confidence interval computation methods available, but one particular approach which I prefer is Bayesian: Use a Beta prior disitribution and use that to obtain a posterior distribution (Beta-Binomial). For example, using the Uniform (Uninformed) prior distribution Beta(1,1), the confidence interval (or "credible interval") is the much more plausible [0.480.94].

One of the nice properties of this approach is the subjective knowledge that is inherently available as input to the model. The hyperparameters alpha and beta can be interpreted as alpha - 1 is the number of successes and beta - 1 is the number of failures. So if we strongly believe a priori that the sample proportion is around 0.9, then we could use the Beta(10, 2) as the prior distribution instead, which gives the credible interval of [0.64, 0.95].

References (2)

References allow you to track sources for this article, as well as articles that were written in response to this article.

Reader Comments (1)

The Beta distribution being a conjugate prior for the Binomial likelihood also seems to make it more plausible as encoding prior knowledge.

June 22, 2013 | Unregistered CommenterDavid

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>