Lesson 4.7: t-Tests for Individual Coefficients

We have our OLS estimates. Now, we must act as judge and jury. This lesson introduces the t-test, the fundamental tool for determining if an estimated coefficient reflects a real, underlying relationship or if it's merely a product of random sampling noise.

Part 1: The Core Question of Inference

We've run our regression and found an estimate, for example, $\hat{\beta}_1 = 0.5$ . This number is our "best guess" from our specific sample.

But a different sample would give a different estimate. The fundamental question of inference is:

Is the true, unobservable effect $\beta_1$ actually zero, and our estimate of 0.5 is just random noise? Or is our estimate reflecting a real, non-zero relationship?

To answer this, we use the formal framework of hypothesis testing.

The Hypothesis Test for a Single Coefficient

Null Hypothesis ( $H_0$ ): The "presumption of innocence." We assume the variable has no effect on $Y$ .
$H_0: \beta_j = 0$
Alternative Hypothesis ( $H_1$ ): The claim we seek evidence for. The variable has a statistically significant effect.
$H_1: \beta_j \ne 0 \quad (\text{Two-sided test})$

Part 2: Constructing the Test Statistic

To test our hypothesis, we need to create a test statistic whose distribution is known when the null hypothesis is true. As we saw in Module 3, the perfect tool for testing a mean when the population variance is unknown is the t-statistic.

The 'Signal-to-Noise' Ratio

The intuition behind the t-statistic is that it forms a signal-to-noise ratio:

t_{\text{stat}} = \frac{\text{Signal}}{\text{Noise}} = \frac{\text{How far our estimate is from zero}}{\text{How much we'd expect our estimate to vary by chance}}

A large t-statistic (e.g., > 2) suggests the signal is strong relative to the random noise, making us doubt the null hypothesis of "no effect."

The Components of the t-statistic

The Signal: This is our point estimate, $\hat{\beta}_j$ .

The Noise (Standard Error): The standard error of our estimate, $\text{se}(\hat{\beta}_j)$ , measures the typical "wobble" or sampling variability of $\hat{\beta}_j$ . To calculate it, we need an estimate of the error variance, $\sigma^2$ .

Deriving the Standard Error

1. Estimate Error Variance: Our unbiased estimator for the error variance $\sigma^2$ is:

\hat{\sigma}^2 = \frac{\text{SSR}}{n - k - 1} = \frac{\mathbf{e}^T\mathbf{e}}{n - k - 1}

where $n$ is the sample size, $k$ is the number of predictors, and $k+1$ is the number of estimated parameters.

2. Estimate the Covariance Matrix of $\bm{\hat{\beta}}$ : We take the true variance formula, $\text{Var}(\bm{\hat{\beta}}) = \sigma^2(\mathbf{X}^T\mathbf{X})^{-1}$ , and plug in our estimate $\hat{\sigma}^2$ :

\hat{\mathbf{V}}(\bm{\hat{\beta}}) = \hat{\sigma}^2 (\mathbf{X}^T\mathbf{X})^{-1}

3. Find the Standard Error: The standard error for a single coefficient $\hat{\beta}_j$ is the square root of the j-th diagonal element of this estimated matrix.

\text{se}(\hat{\beta}_j) = \sqrt{[\hat{\mathbf{V}}(\bm{\hat{\beta}})]_{jj}}

The t-Statistic for a Single Coefficient

t_{\text{stat}} = \frac{\hat{\beta}_j - 0}{\text{se}(\hat{\beta}_j)} \sim t_{n-k-1}

Under the null hypothesis (and the CLM assumptions, including Normality), this statistic follows a t-distribution with $n-k-1$ degrees of freedom.

Part 3: Making the Decision

We use the p-value associated with our calculated $t_{stat}$ to make our decision, following the logic from Module 3.

The p-value Decision Rule

"If the p-value is low, the null must go."

We choose a significance level, $\alpha$ (our threshold for "reasonable doubt," usually 0.05).
We calculate the $p$ -value, which is the probability of getting a t-statistic as extreme as ours, if the null were true.
We compare. If $p\text{-value} < \alpha$ , the result is "statistically significant." The evidence is strong enough to reject the presumption of innocence. We **Reject H₀**.

The Confidence Interval Connection

The t-test is directly linked to the confidence interval for the coefficient:

\text{95% C.I. for } \beta_j = \left[ \hat{\beta}_j - t_{crit} \cdot \text{se}(\hat{\beta}_j), \quad \hat{\beta}_j + t_{crit} \cdot \text{se}(\hat{\beta}_j) \right]

The two methods are perfectly equivalent:

Rejecting $H_0: \beta_j = 0$ at the 5% significance level is mathematically identical to finding that the 95% confidence interval for $\beta_j$ **does not contain zero**.

What's Next? Testing the Whole Model

The t-test is our precision tool for examining one coefficient at a time. It tells us if a single variable is a significant predictor.

But what if we want to ask a bigger question? For example, are *any* of our variables useful? Is our entire model better than just predicting the average of Y? Or, are a specific *group* of variables (e.g., all the variables related to company size) jointly significant?

To answer these questions about multiple coefficients at once, we need a different tool: the **F-test**.

Lesson 4.6: The Gauss-Markov Theorem and the BLUE Property

Lesson 4.8: F-Tests for Joint Hypotheses