Introduction IX – Joint Distributions

In the past we always worked with just one random variable at a time. Unfortunately that is not always what we want to do. Although unfortunately doesn’t quite fit here as it actually becomes more interesting when we observe two or more random variables at the same time, it was what I first thought. I was very scared that it’ll become overwhelming complicated with two or more variables but that’s not the case. Sure it becomes more complicated but also more interesting and after a while the complicated things doesn’t seem to be complicated anymore. The key is to try to understand it and not just memorise it. I encourage all readers to not just rely on my blog-posts but also to read texts about the same topic from other sources. It will help to deepen your understanding.

Given that short – or not so short – introduction, let us talk about what we will do today. The topic for today are joint distributions.

Joint Distributions

What are joint distributions? Joint distributions are basically distributions of two or more random variables, like the height and weight of an animal. Joint distributions allow us to compute probabilities involving two or more random variables and they help us to understand the relationship between the random variables. This all works fine if the random variables are independent. If they are not, we use covariance and correlation to get a picture of the relationship of the random variables. Covariance and Correlation will be the topic of the next Introduction blog-post.

Discrete case

If the random variables are discrete, the joint distribution has a joint probability mass function (joint PMF). The joint PMF p(x_{ i },y_{ j }) gives then the probability for the joint outcome: X=x_{ i }\;and\;Y=y_{ j } .

The joint PMF can be visualised with a joint probability table:

X/Y Y_{ 1 } Y_{ 2 } Y_{ 3 } Y_{ j }
X_{ 1 } p(x_{ 1 },y_{ 1 }) p(x_{ 1 },y_{ 2 }) p(x_{ 1 },y_{ 3 }) p(x_{ 1 },y_{ j })
X_{ 2 } p(x_{ 2 },y_{ 1 }) p(x_{ 2 },y_{ 2 }) p(x_{ 2 },y_{ 3 }) p(x_{ 2 },y_{ j })
X_{ 3 } p(x_{ 3 },y_{ 1 }) p(x_{ 3 },y_{ 2 }) p(x_{ 3 },y_{ 3 }) p(x_{ 3 },y_{ j })
X_{ i } p(x_{ i },y_{ 1 }) p(x_{ i },y_{ 2 }) p(x_{ i },y_{ 3 }) p(x_{ i },y_{ j })

Example: Suppose we roll two dice. Let X be the outcome of the first die and Y the outcome of the second die.

X/Y 1 2 3 4 5 6
1 \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 }
2 \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 }
3 \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 }
4 \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 }
5 \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 }
6 \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 } \frac{ 1 }{ 36 }

Properties of joint PMFs:

  1.  0\le p(x_{ i },y_{ j }) \le 1
  2. \sum_{ i=1 }^{ n }{ \sum_{ j=1 }^{ m }{ p(x_{ i },y_{ j }) } }=1

Continuous case

One might expect that the continuous case is more difficult, but it isn’t. It is basically the same. We just change PMF to PDF, discrete sets to ranges and sums to integrals. The joint probability density function (joint PDF) f(x,y) gives then the probability density at (x, y).

The joint distribution of continuous random variables can be visualised by a big rectangle which contains all outcomes X and Y can take, and a small rectangle which is an event, like 2<X<3, 1<Y<2.

jdc

Properties of joint PDFs:

  1. 0 \le f(x,y)
  2. \int_{ c }^{ d }{ \int_{ a }^{ b }{ f(x,y)dxdy } }=1

Joint Cumulative Distribution Function (CDF)

The joint CDF is given by F(x,y)=P(X\le x, Y\le y) and calculated by either double sums or double integrals:

Continuous case:

F(x,y)=\int_{ c }^{ y }{ \int_{ a }^{ x }{ f(u,v) du dv } }

Discrete case:

F(x,y) = \sum_{ y_{ j }\le y }{\sum_{ x_{ i }\le x }{ p(x_{ i },y_{ j }) }}

Marginal Distribution

Sometimes we want to consider just one of the random variables. We then use the marginal distribution. So the PMF for X without Y is therefore the marginal PMF of X and often denoted p_{X}(x_{ i }) . To get the marginal PMF we just sum up every row or every column. In the continuous case we take the integral of all except of one random variable over the complete ranges of the other random variables.

  • p_{X}(x_{ i })=\sum_{ j }{p(x_{ i }, y_{ j })}
  • p_{Y}(y_{ j })=\sum_{ i }{p(x_{ i }, y_{ j })}
  • f_{X}(x)=\int_{ c }^{ d }{ f(x,y)dy } if [c,d] is the range of Y
  • f_{Y}(y)=\int_{ a }^{ b }{ f(x,y)dx } if [a,b] is the range of X
  • F_{ X }(x)=F(x,d) if d is the upper end of the range of Y
  • F_{ Y }(y)=F(b,y) if b is the upper end of the range of X

Independence

As already mentioned in the introduction, we use covariance and correlation when the random variables are not independent. But how do we check if the random variables are independent? That’s surprisingly easy. Jointly distributed random variables are independent if:

  • F(x,y)=F_{ X }(x)F_{ Y }(y)
  • p(x_{ i }, y_{ j })=p_{ X }(x_{ i })p_{ Y }(y_{ j })
  • f(x,y)=f_{ X }(x)f_{ Y }(y)

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s