# Bayesian Inference VII – Bayesian Updating Continuous Priors and Data

We finally got to the part of Bayesian Updating when both data and prior are continuous. It is, like always, not very different from Bayesian Updating with discrete data and priors except of that we use the PDFs and not the PMFs.

## Continuous Data, Continuous Prior

When we use probability density functions we have to be careful, as the function itself doesn’t give us the probability. It is $f(x|\theta)dx$ that gives us the probability. We therefore have to use $dx$ and $d\theta$. This can be messy sometimes, so we often drop it.

 Hypothesis Prior Likelihood unnormalised Posterior Posterior $\theta$ $f(\theta)$ $f(x|\theta)$ $f(x|\theta)f(\theta)$ $f(\theta|x)=\frac{ f(x|\theta)f(\theta) }{ f(x) }$ $\theta \pm \frac{ d\theta }{ 2 }$ $f(\theta)d\theta$ $f(x|\theta)dx$ $f(x|\theta)f(\theta)dxd\theta$ $f(\theta|x)=\frac{ f(x|\theta)f(\theta)dxd\theta }{ f(x)dx }=\frac{ f(x|\theta)f(\theta)d\theta }{ f(x) }$ Total 1 $f(x)dx=(\int{ f(x|\theta)f(\theta)d\theta })dx$ 1

Example: Suppose we have a normal distributed hypothesis and normal distributed data. So both data and hypothesis are continuous. Suppose data x is drawn from a normal distribution with unknown mean $\theta$ and $\sigma=1$.
Suppose further that $\theta\sim N(2,1)$

What is the posterior PDF?

Answer: Our prior PDF is given by $f(\theta)=\frac{ 1 }{ \sqrt( 2\pi ) }exp^{ \frac{ -(\theta - 2)^{ 2 } }{ 2 } }$ and our likelihood PDF is given by $f(x|\theta)=\frac{ 1 }{ \sqrt( 2\pi ) }exp^{ \frac{ -(x - \theta)^{ 2 } }{ 2 } }$.

We need the product of our prior and likelihood for the unnormalised posterior so we can do these calculations first:

prior x likelihood = $\frac{ 1 }{ \sqrt( 2\pi ) }exp^{ \frac{ -(\theta - 2)^{ 2 } }{ 2 } } \cdot \frac{ 1 }{ \sqrt( 2\pi ) }exp^{ \frac{ -(x - \theta)^{ 2 } }{ 2 } } = \frac{ 1 }{ 2\pi }exp^{ \frac{ -(2\theta^{ 2 }-(4+2x)\theta +4+x^{ 2 }) }{ 2 } }=\frac{ 1 }{ 2\pi }exp^{ \frac{ -(2(\theta^{ 2 }-(2+x)\theta) +4+x^{ 2 }) }{ 2 } }=\frac{ 1 }{ 2\pi }exp^{ \frac{ -(2(\theta-(1+\frac{ x }{ 2 }))^{ 2 } -(1+\frac{ x }{ 2 })^{ 2 } +4+x^{ 2 }) }{ 2 } }= \frac{ 1 }{ 2\pi }exp^{ \frac{ -(-(1+\frac{ x }{ 2 })^{ 2 } +4+x^{ 2 }) }{ 2 } } exp^{ \frac{ -2(\theta-(1+\frac{ x }{ 2 }))^{ 2 } }{ 2 } }=c_{ 1 }exp^{ -(\theta-( 1+\frac{ x }{ 2 } ))^{ 2 } }$

x is a constant because we know the data.

If we now use bayesian updating we get the posterior PDF $c_{ 2 }exp^{ -(\theta-( 1+\frac{ x }{ 2 } ))^{ 2 } }$, what looks like a normal distribution with $\mu_{ post. }=1+\frac{ x }{ 2 }$ and $\sigma^{ 2 }=\frac{ 1 }{ 2 }$. Furthermore we know the normalising constant for normal distributions: $\frac{ 1 }{ \sigma \sqrt{ 2\pi } }$

If we use 4.7 as our data, we get the following graphs:

The prior got pulled towards the data.

NOTE: I used the following python code for the graphs.

from scipy import stats as s
import numpy as np
import matplotlib.pyplot as plt
import math

data = 4.7

def prior(xs):
ys  = []
for x in xs:
y = math.exp((-1*(x-2)**2)/2)/(math.sqrt(2*math.pi))
ys.append(y)
return ys

def posterior(xs):
ys = []
for x in xs:
y = math.exp(-(x-(1+data/2))**2)/(math.sqrt(math.pi))
ys.append(y)
return ys

prior_x = [x for x in np.arange(-6,6,0.1)]
prior_y = [y for y in prior(prior_x)]

posterior_x = [x for x in np.arange(-6,6,0.1)]
posterior_y = [y for y in posterior(posterior_x)]

fid = plt.figure()
ax1 = plt.subplot2grid((1,1), (0,0))
ax1.plot(prior_x, prior_y, color='blue', label='prior')
ax1.plot(posterior_x, posterior_y, color='red', label='posterior, data={}'.format(data))

ax1.set_yticks(np.arange(0, 1.1, 0.1))
plt.legend()
plt.show()