Pratidin
2026 · Statistics Optional Paper II · 10 marksMainsQuestion 5c

For the General Linear Model
Y = Xβ + u
where u ~ N(0, σ²Iₙ)
obtain the Maximum Likelihood Estimator (MLE) β̂ of β. Further, let e = (Y − Xβ̂), and X is a matrix of order n × k with rank of X, k < n. If e′e = u′Mu, where M = (I − X(X′X)⁻¹X′), show that e′e/σ² follows chi-square distribution with (n − k) degrees of freedom (d.f.).

10 marks · directive: Show

Answer

The core of a good answer

Maximising the normal log-likelihood −(n/2) log(2πσ²) − (Y − Xβ)′(Y − Xβ)/(2σ²) gives β̂ = (X′X)⁻¹X′Y (identical to OLS) and σ̂² = e′e/n; since M is symmetric and idempotent with trace n − k, writing M = PΛP′ with Λ = diag(1, ..., 1, 0, ..., 0) and w = P′u/σ ~ N(0, Iₙ) gives e′e/σ² = u′Mu/σ² = w₁² + ... + wₙ₋ₖ² ~ χ²(n − k).

An outline written by upsc.help, not an official UPSC answer; every fact checked by a separate review. How we work

What the question asks

Two parts: derive the MLE by differentiating the log-likelihood, then prove the distribution of the residual sum of squares using the properties of M (symmetric, idempotent, rank n − k) and the theorem on idempotent quadratic forms in normal variables.

Answer outline

Introduction

Y is n × 1, X is n × k fixed with rank k, β is k × 1, u ~ Nₙ(0, σ²I). So Y ~ Nₙ(Xβ, σ²I).

Step 1: Likelihood

Step 2: MLE of β and σ²

Step 3: e′e = u′Mu

Step 4: Rank of M

Step 5: Distribution

Conclusion

β̂_MLE = (X′X)⁻¹X′Y and e′e/σ² ~ χ²(n − k); hence E(e′e) = (n − k)σ², so s² = e′e/(n − k) is unbiased, and since MX = 0 makes e independent of β̂, this result underlies the t and F tests in regression.

Examples and data to cite

Facts

Facts to remember

Sources used for the explanation

Write a timed answer in the app