Discussing Gauss-Markov linear model Y_{n×1} = X_{n×p} β_{p×1} + u_{n×1} with E(Y) = Xβ and D(Y) = variance and covariance matrix of Y = σ²I, show that
E[Y′{I − X(X′X)⁻¹X′}Y] = σ²(n − p)
10 marks · directive: Show
The core of a good answer
With M = I − X(X′X)⁻¹X′ symmetric and idempotent, E(Y′MY) = σ² tr(M) + β′X′MXβ; since MX = 0 and tr(M) = n − tr[(X′X)⁻¹X′X] = n − p, E(Y′MY) = σ²(n − p), so the residual sum of squares divided by n − p is unbiased for σ².
An outline written by upsc.help, not an official UPSC answer; every fact checked by a separate review. How we work
What the question asks
After briefly stating the model and its assumptions (linear mean, homoscedastic uncorrelated errors, X of full column rank p), the proof is the expectation of a quadratic form. The key facts are MX = 0 and the trace identity tr(AB) = tr(BA).
Answer outline
Introduction
Gauss-Markov model: Y = Xβ + u, E(u) = 0, D(u) = σ²I, rank X = p (so X′X is invertible). H = X(X′X)⁻¹X′ is the hat matrix; M = I − H; Y′MY is the residual sum of squares.
Step 1: The model
- β̂ = (X′X)⁻¹X′Y is the least squares estimator and, by the Gauss-Markov theorem, the BLUE of β.
- Residual vector e = Y − Xβ̂ = MY; SSE = e′e = Y′M′MY = Y′MY.
Step 2: Properties of M
- M′ = M; M² = I − 2H + H² = M since H² = X(X′X)⁻¹X′X(X′X)⁻¹X′ = H.
- MX = X − X(X′X)⁻¹X′X = 0.
Step 3: Expectation of the quadratic form
- E(Y′MY) = tr[M D(Y)] + E(Y)′M E(Y) = σ² tr(M) + β′X′MXβ = σ² tr(M).
Step 4: Trace
- tr(H) = tr[(X′X)⁻¹X′X] = tr(I_p) = p.
- tr(M) = n − p.
Step 5: Result
- E[Y′{I − X(X′X)⁻¹X′}Y] = σ²(n − p).
- Hence σ̂² = SSE/(n − p) is unbiased for σ².
Conclusion
The residual sum of squares has expectation σ²(n − p), which justifies dividing by the error degrees of freedom n − p. No normality is needed.
Examples and data to cite
- Simple linear regression has p = 2, so the error mean square is SSE/(n − 2).
Facts to remember
- E(Y′AY) = tr(AΣ) + μ′Aμ.
- Hat matrix H = X(X′X)⁻¹X′ is symmetric idempotent with trace p.
- For an idempotent matrix, rank = trace.
- tr(AB) = tr(BA).