Bayesian Inference V – Bayesian Updating Continuous Priors

We already know how to do Bayesian Updating with discrete priors. Today we will learn how to do Bayesian Updating with continuous priors.

Continous Priors

To do Bayesian Updating with continuous priors but with discrete data – we will look at the case that both is discrete next time – we just change sums to integrals and PMFs to PDFs.

We often denote the hypothesis \theta “theta” where theta is a range and not a discrete hypothesis. The hypothesis is then often written as: H:\;\theta \pm \frac{ d\theta }{ 2 } , what means that the hypothesis is in a small interval of width d\theta around \theta . That is because the probability is given by f(\theta)d\theta

If we apply all of the above mentioned things we get a Bayesian Updating table that looks like this:

Hypothesis Prior Likelihood unnormalised Posterior Posterior
\theta \pm \frac{ d\theta }{ 2 } f(\theta)d\theta p(x|\theta) p(x|\theta)f(\theta)d\theta \frac{ 1 }{ T }p(x|\theta)f(\theta)d\theta
Total \int_{ a }^{ b }{ f(\theta)d\theta }=1 T=\int_{ a }^{ b }{p(x|\theta)f(\theta)d\theta } 1

p(x) which is the total of the unnormalised posterior column can be hard to calculate and we often use a computer to do so, in case we don’t have a conjugate prior. More about conjugate priors soon.

Example: Suppose we have a coin with flat prior and unknown probability p of heads. We flip the coin and get heads. What is the posterior PDF?

Answer: A flat prior means that f(\theta)=1 on the interval [0,1] which mens that all hypotheses are equally likely and that the hypotheses are between 0 and 1. Furthermore we let x=1 mean that the outcome was heads.

The definition of our hypothesis tells us that p(x|\theta)=\theta because our probability of getting heads is our hypothesis. We then have the following Bayesian Updating table:

Hypothesis Prior Likelihood unnormalised Posterior Posterior
\theta \pm \frac{ d\theta }{ 2 } 1d\theta \theta  \theta d\theta 2\theta d\theta 
Total 1 T=\int_{ 0 }^{ 1 }{ \theta d\theta } =\frac{ 1 }{ 2 } 1

Our posterior PDF after seeing one heads is then: f(\theta|x)=2\theta

We can see that Bayesian Updating with continuous priors doesn’t differ much from Bayesian Updating with discrete priors.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s