Let X₁, X₂, ⋯, Xₙ be i.i.d. random variables following N(μ, σ²). Find the maximum likelihood estimator for σ² and examine whether the same is unbiased. If biased, suggest an unbiased estimator for σ² and examine whether it is consistent. Also compare the mean squared errors of the estimators.
20 marks · directive: Find
The core of a good answer
The MLE is σ̂² = (1/n)∑(Xᵢ − X̄)², with E(σ̂²) = (n − 1)σ²/n, so it is biased; S² = ∑(Xᵢ − X̄)²/(n − 1) is unbiased and consistent (V(S²) = 2σ⁴/(n − 1) → 0); MSE(σ̂²) = (2n − 1)σ⁴/n² is smaller than MSE(S²) = 2σ⁴/(n − 1) by (3n − 1)σ⁴/[n²(n − 1)].
An outline written by upsc.help, not an official UPSC answer; every fact checked by a separate review. How we work
What the question asks
A full classical treatment: derive the MLE from the log-likelihood (μ unknown), compute its expectation using ∑(Xᵢ − X̄)²/σ² ~ χ²ₙ₋₁, correct the bias, prove consistency, and compare MSE = variance + bias². The answer should say clearly that unbiasedness and smaller MSE point in different directions here.
Answer outline
Introduction
log L(μ, σ²) = −(n/2) log(2πσ²) − ∑(xᵢ − μ)²/(2σ²). Key fact: ∑(Xᵢ − X̄)²/σ² ~ χ²ₙ₋₁, with mean n − 1 and variance 2(n − 1).
Step 1: MLE
- ∂ log L/∂μ = 0 ⇒ μ̂ = X̄.
- ∂ log L/∂σ² = −n/(2σ²) + ∑(xᵢ − μ)²/(2σ⁴) = 0 ⇒ σ̂² = (1/n)∑(Xᵢ − X̄)².
Step 2: Bias
- E(σ̂²) = (σ²/n)(n − 1) = σ² − σ²/n ≠ σ², so biased; bias = −σ²/n → 0 (asymptotically unbiased).
Step 3: Unbiased estimator
- S² = nσ̂²/(n − 1) = ∑(Xᵢ − X̄)²/(n − 1); E(S²) = σ².
Step 4: Consistency
- V(S²) = σ⁴ × 2(n − 1)/(n − 1)² = 2σ⁴/(n − 1) → 0.
- Unbiased with variance → 0, so by Chebyshev S² → σ² in probability: consistent. (σ̂² is consistent too.)
Step 5: MSEs
- MSE(S²) = V(S²) = 2σ⁴/(n − 1).
- V(σ̂²) = 2(n − 1)σ⁴/n²; MSE(σ̂²) = 2(n − 1)σ⁴/n² + σ⁴/n² = (2n − 1)σ⁴/n².
- MSE(S²) − MSE(σ̂²) = (3n − 1)σ⁴/[n²(n − 1)] > 0 (simplified in Python).
Step 6: Remark
- Among estimators c∑(Xᵢ − X̄)², the MSE is least for c = 1/(n + 1).
Conclusion
The MLE of σ² is biased but has the smaller mean squared error; S² is unbiased and consistent. Both MSEs tend to 0 and their ratio tends to 1 as n grows.
Examples and data to cite
- Example n = 10: MSE(σ̂²) = 0.19σ⁴, MSE(S²) = 0.222σ⁴.
Facts to remember
- Normal MLEs: μ̂ = X̄, σ̂² = ∑(Xᵢ − X̄)²/n.
- ∑(Xᵢ − X̄)²/σ² ~ χ²ₙ₋₁ (Cochran's theorem); E = n − 1, V = 2(n − 1).
- MSE = variance + bias².
- An estimator whose bias and variance both → 0 is consistent.
- Bessel's correction: dividing by n − 1 removes the bias.