Lesson 4.5: The Classical Linear Model (CLM) Assumptions

This lesson forms the theoretical bedrock of the entire module. We will rigorously define the set of assumptions under which the Ordinary Least Squares (OLS) estimator is guaranteed to have desirable properties. These assumptions are the 'rules of the game' that, if met, allow us to prove that OLS is the 'Best Linear Unbiased Estimator' (BLUE) via the Gauss-Markov Theorem.

Part 1: The 'BLUE' Promise

We have derived the OLS estimator formula, $\bm{\hat{\beta}} = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}$ . But why should we use this estimator over any other? The answer lies in the **Gauss-Markov Theorem**, which states that if a set of assumptions holds, OLS is **BLUE**.

Deconstructing 'BLUE'

BLUE is an acronym for the **Best Linear Unbiased Estimator**. It is the gold standard for estimators in econometrics.

Best: 'Best' means having the minimum variance among all estimators in its class. It is the most precise, or most efficient.
Linear: The estimator is a linear function of the dependent variable, $\mathbf{y}$ . We can see this in the formula: $\bm{\hat{\beta}} = \mathbf{Ay}$ , where $\mathbf{A} = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T$ .
Unbiased: The expected value of the estimator is the true parameter: $E[\bm{\hat{\beta}}] = \bm{\beta}$ . On average, our estimates are correct.
Estimator: It is a rule that tells us how to use data to estimate an unknown parameter.

The Gauss-Markov Theorem is a contract: if your model and data satisfy the following assumptions, then OLS is the best possible linear unbiased estimator. We will now detail these assumptions with extreme care.

Part 2: The Gauss-Markov Assumptions (The Rules of the Game)

The first five assumptions are collectively known as the Gauss-Markov assumptions. They are sufficient to prove that OLS is BLUE.

Assumption 1: Linearity in Parameters

Mathematical Statement: The model is linear in the parameters $\bm{\beta}$ and the error term $\bm{\epsilon}$ .

\mathbf{y} = \mathbf{X}\bm{\beta} + \bm{\epsilon}

Explanation: This assumption dictates the functional form of the model. It means the parameters are simple coefficients and are not, for example, squared ( $\beta^2$ ) or used as exponents ( $e^\beta$ ). It does *not* mean the relationship must be a straight line. A model like $Y = \beta_0 + \beta_1 X + \beta_2 X^2 + \epsilon$ is still linear in parameters, even though it describes a parabola.

Consequence of Violation: If the true relationship is non-linear in parameters, the OLS estimator is fundamentally misspecified. It will be biased and inconsistent, describing a relationship that does not exist.

Assumption 2: Full Rank (No Perfect Multicollinearity)

Mathematical Statement: The $n \times (k+1)$ design matrix $\mathbf{X}$ has full column rank. This means $\text{rank}(\mathbf{X}) = k+1$ and the matrix $(\mathbf{X}^T\mathbf{X})$ is invertible.

Explanation: This means none of the independent variables is a perfect linear combination of the others. There is no redundant information. For example, you cannot include a variable for "height in inches" and another variable for "height in centimeters" in the same model, as one is a perfect multiple of the other.

Consequence of Violation: If perfect multicollinearity exists, the inverse $(\mathbf{X}^T\mathbf{X})^{-1}$ cannot be computed. The OLS estimator $\bm{\hat{\beta}}$ is not uniquely defined. Statistical software will either crash or automatically drop one of the redundant variables.

Assumption 3: Exogeneity of the Independent Variables

Mathematical Statement: The conditional mean of the error term, given all values of the independent variables, is zero.

E[\bm{\epsilon} | \mathbf{X}] = \mathbf{0}

Explanation: This is the most important and most frequently violated assumption. It means that the unobserved factors that affect $Y$ (and are contained in $\bm{\epsilon}$ ) are, on average, uncorrelated with our observed predictors $\mathbf{X}$ . There is no systematic relationship between what we can see and what we can't see.

Consequence of Violation: If exogeneity is violated (a condition called **endogeneity**), the OLS estimator $\bm{\hat{\beta}}$ becomes **biased and inconsistent**. It will never converge to the true $\bm{\beta}$ , even with infinite data. The coefficients are meaningless and cannot be interpreted as causal effects. This is the most catastrophic failure of a regression model.

Common Cause: Omitted Variable Bias. If we model `wage` as a function of `education`, but omit the variable `innate ability`, this assumption is violated because `ability` is in the error term but is also correlated with `education`.

Assumption 4: Homoskedasticity and No Autocorrelation

Mathematical Statement: The variance-covariance matrix of the errors, conditional on $\mathbf{X}$ , is a spherical error covariance matrix.

E[\bm{\epsilon}\bm{\epsilon}^T | \mathbf{X}] = \sigma^2 \mathbf{I}

Explanation: This assumption has two parts:

Homoskedasticity: The diagonal elements are constant. $\text{Var}(\epsilon_i|\mathbf{X}) = \sigma^2$ for all $i$ . The amount of "noise" is the same for all observations.
No Autocorrelation: The off-diagonal elements are zero. $\text{Cov}(\epsilon_i, \epsilon_j|\mathbf{X}) = 0$ for $i \ne j$ . The errors are uncorrelated with each other.

Consequence of Violation: If either part is violated (leading to heteroskedasticity or autocorrelation), the OLS estimator $\bm{\hat{\beta}}$ remains **unbiased and consistent**, but it is no longer **Best**. It is not the most efficient estimator. More importantly, the standard formula for the variance of the estimator, $\text{Var}(\bm{\hat{\beta}}) = \sigma^2(\mathbf{X}^T\mathbf{X})^{-1}$ , is **incorrect**. This means all our standard errors, t-tests, and F-tests are invalid.

Part 3: The Normality Assumption (For Inference)

There is a final, optional assumption that is not required for the Gauss-Markov theorem, but is necessary for exact finite-sample inference.

Assumption 5: Normality of Error Terms

Mathematical Statement: The error terms, conditional on $\mathbf{X}$ , are normally distributed.

\bm{\epsilon} | \mathbf{X} \sim \mathcal{N}(\mathbf{0}, \sigma^2\mathbf{I})

Explanation: This assumption combines the exogeneity and spherical error assumptions and adds that the distribution of the unobserved factors follows a bell curve.

Consequence: If this assumption holds, then the OLS estimator $\bm{\hat{\beta}}$ is also normally distributed in finite samples. This means our t-tests and F-tests are **exact**, not just asymptotically valid. Without this assumption, we must rely on the Central Limit Theorem and large sample sizes for our inference to be approximately correct.

Summary of the Contract

If Assumptions 1-4 hold (Gauss-Markov conditions), then OLS is the **Best Linear Unbiased Estimator (BLUE)**.
If Assumptions 1-5 hold (the full CLM), then our t-statistics and F-statistics have exact t and F distributions, respectively, in any sample size.
Violating Assumption 3 (Exogeneity) is the most severe failure, rendering our coefficient estimates meaningless.
Violating Assumption 4 (Homoskedasticity/No Autocorrelation) is less severe but invalidates our standard errors and statistical tests, requiring correction.

What's Next? Proving the Promise

We have laid out the complete set of rules for the Classical Linear Model. We have stated that if these rules are followed, OLS is the "Best Linear Unbiased Estimator."

But a statement is not a proof. In the next lesson, we will undertake the formal, 'no-skip' mathematical proof of the **Gauss-Markov Theorem**, demonstrating with matrix algebra that under these assumptions, no other linear unbiased estimator can have a smaller variance than OLS.

Up Next: The Proof of BLUE: The Gauss-Markov Theorem