Let l′β be any estimable linear function of parameters β₁, β₂, ⋯, βₚ. Then prove that—
(i) there exists a unique linear function c′Y of random variables Y₁, Y₂, ⋯, Yₙ such that c ∈ V(A′) and E(c′Y) = l′β, where V(A′) is the vector space generated by row vectors of A′ and E(Y) = Aβ;
(ii) variance of c′Y is less than the variance of any other linear unbiased estimator.
20 marks · directive: Prove
The core of a good answer
Since l′β is estimable, some a′Y has E(a′Y) = l′β, i.e. A′a = l; projecting a onto the column space of A (the space spanned by the rows of A′) gives c with A′c = l, unique because two such vectors differ by a vector both in and orthogonal to that space; any other unbiased b′Y has b = c + d with A′d = 0, so V(b′Y) = σ²(c′c + d′d) > σ²c′c unless b = c.
An outline written by upsc.help, not an official UPSC answer; every fact checked by a separate review. How we work
What the question asks
This is the Gauss-Markov theorem in the possibly less-than-full-rank model, written in terms of the estimating space. The proof needs: the meaning of estimability, the orthogonal decomposition Rⁿ = V(A′) ⊕ V(A′)^⊥, and the dispersion assumption D(Y) = σ²I (implicit in the model, to be stated).
Answer outline
Introduction
Model: E(Y) = Aβ, D(Y) = σ²I, A of order n × p, any rank. V(A′) = column space of A ⊂ Rⁿ (spanned by the rows of A′). Its orthogonal complement is {d : A′d = 0}.
Step 1: Estimability
- l′β is estimable if some a′Y satisfies E(a′Y) = a′Aβ = l′β for all β, i.e. A′a = l (l lies in the row space of A).
Step 2: Existence of c
- Decompose a = c + d with c ∈ V(A′), d ⊥ V(A′), so A′d = 0.
- Then A′c = A′a = l, so E(c′Y) = c′Aβ = l′β.
Step 3: Uniqueness
- If c₁, c₂ ∈ V(A′) both satisfy A′cᵢ = l, then A′(c₁ − c₂) = 0, so c₁ − c₂ ⊥ V(A′).
- But c₁ − c₂ ∈ V(A′); a vector orthogonal to itself is 0, so c₁ = c₂.
Step 4: Minimum variance
- Let b′Y be any linear unbiased estimator of l′β: A′b = l. Put d = b − c; then A′d = 0, so d ⊥ c (c ∈ V(A′)).
- V(b′Y) = σ²b′b = σ²(c + d)′(c + d) = σ²(c′c + d′d) ≥ σ²c′c = V(c′Y).
- Equality only if d = 0, i.e. b = c; so c′Y has strictly smaller variance than any other linear unbiased estimator.
Step 5: Link with least squares
- c′Y = l′β̂ for any solution β̂ of the normal equations A′Aβ̂ = A′Y, so the BLUE is the least squares estimator.
- V(l′β̂) = σ²c′c = σ² l′(A′A)⁻l for any generalised inverse.
Conclusion
For every estimable l′β there is exactly one unbiased linear estimator with coefficient vector in the estimation space, and it is the BLUE (Gauss-Markov theorem). Any other unbiased linear estimator adds an uncorrelated 'error-space' component d′Y that only increases variance.
Examples and data to cite
- One-way ANOVA: treatment contrasts ∑cᵢτᵢ with ∑cᵢ = 0 are estimable and their BLUE is ∑cᵢȳᵢ, although the individual τᵢ are not estimable.
Facts to remember
- l′β estimable ⇔ l ∈ row space of A ⇔ l′ = a′A for some a.
- Rⁿ = C(A) ⊕ N(A′), orthogonal decomposition.
- V(b′Y) = σ²b′b when D(Y) = σ²I.
- Gauss-Markov theorem: least squares estimators of estimable functions are BLUE.