Bayesian Inference VI – Beta Distribution

We’ve seen last time how Bayesian Updating with continuous priors works. As mentioned back then, it isn’t always easy to calculate the total probability of X (p(x)). Today I will introduce the Beta Distribution which makes these calculations easier.

Beta Distribution

The beta(a,b) distribution is a two parameter distribution on range [0,1] and is therefore ideal to model unknown probabilities:

$f(\theta)=cx^{ a-1 }(1-x)^{ b-1 }$ where c is the normalising constant: $\frac{ (a+b-1)! }{ (a-1)!(b-1)! }$

Furthermore the beta distribution is the conjugate prior for Bernoulli, Binomial and Geometric distributions. Conjugate prior means that if our prior is beta our posterior will also be beta.

Example: Suppose we have a coin with unknown probability p of heads. We toss the coin 20 times and get 15 heads and 5 tails. What is the posterior PDF?

 Hypothesis Prior Likelihood unnormalised Posterior Posterior $\theta \pm \frac{ d\theta }{ 2 }$ $1d\theta$ $c_{ 1 }\theta^{ 15 }(1-\theta)^{ 5 }$ $c_{ 1 }\theta^{ 15 }(1-\theta)^{ 5 }d\theta$ $c_{ 2 }\theta^{ 15 }(1-\theta)^{ 5 }$ Total 1 $T=\int_{ 0 }^{ 1 }{ c_{ 1 }\theta^{ 15 }(1-\theta)^{ 5 } }$ 1

We can see that the posterior PDF is beta(16,6).