Bayesian Inference VI – Beta Distribution

We’ve seen last time how Bayesian Updating with continuous priors works. As mentioned back then, it isn’t always easy to calculate the total probability of X (p(x)). Today I will introduce the Beta Distribution which makes these calculations easier.

Beta Distribution

The beta(a,b) distribution is a two parameter distribution on range [0,1] and is therefore ideal to model unknown probabilities:

f(\theta)=cx^{ a-1 }(1-x)^{ b-1 } where c is the normalising constant: \frac{ (a+b-1)! }{ (a-1)!(b-1)! }

Furthermore the beta distribution is the conjugate prior for Bernoulli, Binomial and Geometric distributions. Conjugate prior means that if our prior is beta our posterior will also be beta.

Example: Suppose we have a coin with unknown probability p of heads. We toss the coin 20 times and get 15 heads and 5 tails. What is the posterior PDF?

Hypothesis Prior Likelihood unnormalised Posterior Posterior
\theta \pm \frac{ d\theta }{ 2 } 1d\theta c_{ 1 }\theta^{ 15 }(1-\theta)^{ 5 } c_{ 1 }\theta^{ 15 }(1-\theta)^{ 5 }d\theta c_{ 2 }\theta^{ 15 }(1-\theta)^{ 5 }
Total 1 T=\int_{ 0 }^{ 1 }{ c_{ 1 }\theta^{ 15 }(1-\theta)^{ 5 } } 1

We can see that the posterior PDF is beta(16,6).


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s