Queens University at Kingston


glossary File 4
[glossary Contents] [Previous File] [Next File]

D. Joint Distributions

Let X and Y be two random variables defined on the same sample space.


Sample Space
all undergraduate at Queen's
amount of outside study time put in by student (per day)
overall average marks of the student

The Joint Cumulative Distribution of X and Y is defined as F(x,y) = P(X < = x & Y < = y)

Example (continued)
F(24,100) = 1
all students study less than 24 hours a day and all have averages less than or equal to 100
F(0,0) = 0
no students study less than 0 hours and have averages less than or equal to 0.

Joint density of X and Y is defined as f(x,y) = Prob( X= x & Y = y) [discrete] and f(x,y) = d² F(x,y) / dxdy [continuous]

The marginal density function for X is $f_X(x) \equiv \int_{-\infty}^{\infty}f(x,y)dy$

Notice that the marginal density is the same as the ordinary density for the single randome variable. The other random variable Y is integrated out (doesn't matter what value it takes on).

X and Y are (statistically) Independent if and only if
$$f(x,y) = f_X(x) f_Y (y)$$
for all values of x and y

In words: if the joint density factors into the product of the two marginal densities then X and Y are said to be independent.

Joint Expectations: Given a joint density function f(x,y) and a function g(x,y) that is a function of the two random variables X and Y, the expected value of g(x,y) is defined as
$$E[g(x,y)] \equiv \int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty} g(x,y)f(x,y)dxdy$$

Examples of joint expectations:

  1. The mean of one of the random variables can be written using the joint density function:
    $$E[Y] = \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} yf(x,y)dxdy = \int_{-\infty}^{+\infty}yf_Y(y)dy.$$
    This is the case where g(x,y) = y.
  2. Covariance is defined as E[ (X-E[X])(Y-E[Y]) ]
    Covariance extends The notion of variance to two dimensions. Its value measures the linear relationship between r.v. Positive covariance indicates that when X is above its mean then Y tends to be above its mean as well - positive statistical relationship. Negative covariance indicates that when X is above its mean Y tends to be below its mean - negative statistical relationship
  3. Correlation between two random variables ($\rho$ ) is defined as
    $$\hbox{corr}(X,Y) \equiv {\hbox{Cov}[X,Y] \over \sqrt{Var[X]Var[Y]} }$$
    ($\rho$ is a common synonym for corr).

Properties of Covariance

  1. X and Y independent $\rightarrow$ Cov[X,Y] = 0
  2. Cov[X,Y]
  3. If X and Y are normal random variables then Cov[X,Y] = 0 $\rightarrow$ X and Y independent.
    Note that Cov[X,2Y] = 2Cov[X,Y], so a simple linear change in the random variables will increase or decrease the size of their covariances. Correlation avoids this scaling effect.

    Properties of Correlation

    1. $-1 \le \rho \le 1$
    2. Cov[X,Y] = 0 $\rightarrow$ corr[X,Y] = 0 (Zero covariance is the same as zero correlation)
    3. If X and Y are linear functions of each other, then |corr[X,Y]| = 1.
    4. corr[aX,bY] = corr[X,Y]
      The absolute value of corr[X,Y] measures the degree of linear relationship between them. The sign of corr[X,Y] indicates whether there is a positive or negative relationship between X and Y. Unlike covariance, correlation is unaffected by the scaling of the random variables by constants.

      Example of Joint Distribution

      Let X take on the values 1, 2
      Let Y take on the values 0,1
                                           f (x)
             f(x,y)    0         1          X
            1     |  0.1    |   0.3    |   0.4
                  |         |          |
            2     |  0.4    |   0.2    |   0.6
           f (y)     0.5        0.5        1.00
      Confirm the following
      E[X] = 1.5
      E[Y] = 0.4
      Var(X) = 0.25
      Var(Y) = 0.24
      X and Y are not independent
      E[X | Y=0 ] = 1.8

      You can get as much practice as you want with this kind of example by running the tutorial for week 2.

[glossary Contents] [Next File] [Top of File]

This document was created using HTX, a (HTML/TeX) interlacing program written by Chris Ferrall.
Document Last revised: 1997/1/5