[multiple Contents] [Previous File] [Next File]

- Random matrix
- Expectation of a random matrix V (m x r).
- Variance of a random vector Let v by m x 1. Then Var(v) E[ (v-E[v])(v-E[v])' ] This generalizes the notion of covariance, which itself generalized the notion of variance. Notice that (v-E[v])(v-E[v])' is a m x m matrix (when v is m x 1). So Var(v) is always a square matrix. If we multiply (v-E[v])(v-E[v])' out, we can see that the ith row, jth column of it contains . The expectation of this scalar random variable is by definition:

A random matrix is a matrix of random variables. For example the vector of disturbance terms is a random vector since each element of the vector is a random variable. A constant matrix is, of course, a special type of random matrix in the same sense that a constant is a special case of a random variable that does not vary across points in the sample space.

The expectation of a random matrix is the matrix of expectations of each random
variable in the random vector. That is, let V be a m x r random matrix, and let
denote the random variable in the ith row and jth column of V. Then

Note that expectation is still a linear operator when defined this way.
That is, if A is a n x m matrix of constants and C is a n x r vector of
constants, then

E[AV + C] = AE[V] + C

Notice that m does not have to equal r. That is, the transformation matrix A may change the dimensions of the random matrix. AV is a n x r random matrix, not a m x r. The proof of this result requires you to write out what an element of the matrix AV+C looks like, apply the expectation operator to it, and then see that the result is exactly the same as if you had written out AE[V] + C.

So each element of Var(v) is a covariance. The diagonal elements of Var(v) are simply the variances of the corresponding elements of the v vector. Since Cov(r,s) = Cov(s,r), the matrix Var(v) is

Var[Av] = E[ (Av-AE[v])(Av-AE[v])' ] = E[ A(v-E[v])(v-E[v])'A'] = AVar[v]A'This result generalizes the simpler result that when multiplying a random variable by a constant it changes the variance by the square of the constant. In fact, if m=n=1, then we simply get back the old formula Var(av) = a&149;Var(v).

Here's an example. Let
with

This means that v contains two random variables that have zero covariance.
(They might be statistically independent of each other as well, since
independence implies zero covariance). Let A = (2 -2). Then

is the random variable that is simply twice the difference of the two
random variables in the vector v. From the result above,

This is exactly the formula we would have used without the matrix notation,
since the weighted sum of zero-covariance random variables is the sum
of the variances multiplied by the square of the weights. Keeping
track of variances and covariances in matrices is very convenient.

The matrix X is fixed (non-random) across samples.

E[u] = **0**, where
**0** is a N x 1 vector containing 0s.

Var[u] = I

In words: the variance matrix for u is diagonal (0 covariance across observations) and has equal elements on the diagonal (equal variance).

In words: The u vector is a **normal** vector, in the sense
that each element of u is normally distributed.
Notice that the only change between simple and multiple regression is the change in
the dimension of X. So if we had started out with matrix notation we would have been
able to use these forms of the assumptions from the start. That is, **A4** above is the
same as the **A4**, just using different notation. The same is true of **A3** through **A7**,
the only difference being that **A5** and **A6** can stated together.

Now we can start analyzing the statistical properties of OLS estimate and .

Document Last revised: 1997/1/5