Queens University at Kingston

HyperMetricsNotes

regress File 5
[regress Contents] [Previous File] [Next File]

D. Statistical Properties of OLS Estimates Under LRM Assumptions

We now use the assumptions stated earlier to establish the major statistical properties of OLS estimates within the framework of the LRM. Assumption A6 introduces a new parameter, the variance of the disturbance terms $\sigma^2$ . This is another unknown population parameter which must be estimated from the data. Since the prediction error $e_i$ is analogous to the actual disturbance term $u_i$ , it might make sense to estimate the variance of the u's with the (observable) variance of the prediction errors.

We already know from the first normal equation that the OLS prediction errors have a (sample) mean of 0. Their sample variance equals therefore equals
$$\hat\sigma^2 = \sum_{i=1}^N {e_i^2\over N-2} = {SS\over N-2}$$
The reason for dividing by N-2 rather than N will be made clear later. Intuitively, the reason for dividing by N-2 is that two parameters must be estimated to compute the error terms ($\beta_1$ and $\beta_2$ ). When computing the sample variance of a variable, it is typical to divide by $N-1$ because one parameter (the sample mean) must be estimated to compute the sample variance. This changes the degrees of freedom in the sums of squares that defines $\hat\sigma^2$ .

Each of the following properties is derived in class.

1. Properties Requiring Assumptions A0 through A4
OLS estimates are unbiased estimates of $\beta_1$ , $\beta_2$ , and $\sigma^2$ .
2. Properties Requiring Assumptions A0-A4 and A5 and A6.
The true Variances of the OLS estimates take the form:
$$\eqalign{Var(\hat\beta_2) &= \sigma^2 \sum_{i=1}^N k_i^2 = {\sigma^2 \over \sum x^2_i}\cr Var(\hat\beta_1) &= \sigma^2 {\sum X_i^2 \over N \sum x_i^2}\cr Cov(\hat\beta_1,\hat\beta_2) &= -\bar X \hbox{Var}(\hat\beta_2) = -\bar X {\sigma^2 \over \sum x^2_i} }$$
Since the true variances include the unknown population parameter $\sigma^2$ , it is necessary to estimate the variances by inserting the estimate of $\sigma^2$ :
$$\eqalign{\hat{Var}(\hat\beta_2) &= {\hat\sigma^2 \over \sum x^2_i}\cr \hat{Var}(\hat\beta_1) &= \hat\sigma^2 {\sum X_i^2 \over N \sum x_i^2}\cr}$$
The estimated standard errors are simply defined as the square roots of the respective variances of the OLS estimators:
$$\eqalign{\hat{se}(\hat\beta_2) &= \sqrt{ {\hat\sigma^2 \over \sum x^2_i} }\cr \hat{se}(\hat\beta_1) &= \sqrt{\hat\sigma^2 {\sum X_i^2 \over N \sum x_i^2}}\cr}$$
Gauss-Markov Theorem:
Under Assumptions A0-A6, OLS estimates are the Best (lowest variance) Linear Unbiased Estimates of $\beta_1$ and $\beta_2$ .
3. Properties Requiring Assumptions A0 through A6 and A7
OLS estimates are normally distributed:
$$\hat\beta_1 \sim N\Bigl(\beta_1,Var(\hat\beta_1)\Bigr)$$
$(N-2){\hat\sigma^2\over \sigma^2} \sim \chi^2_{N-2}$
The OLS estimates of $\beta_1$ and $\beta_2$ are statistically independent of $\hat\sigma^2$ .
The two ratios:
$$\eqalign{ {\hat\beta_1 -\beta_1 \over \hat{se}(\hat\beta_1)} &\sim t_{N-2}\cr {\hat\beta_2 -\beta_2 \over \hat{se}(\hat\beta_2)} &\sim t_{N-2}\cr }$$
This property follows directly from the three previous properties and the definition of the t distribution.
[regress Contents] [Next File] [Top of File]

This document was created using HTX, a (HTML/TeX) interlacing program written by Chris Ferrall.
Document Last revised: 1997/1/5