Explain the problem of classification in discriminant analysis. In case of two known multivariate normal populations, define Fisher’s discriminant function and calculate it for the data given below :
μ⁽¹⁾ = (24, 12)′, μ⁽²⁾ = (18, 10)′ and Σ = [9 −5; −5 8] (rows (9, −5) and (−5, 8))
10 marks · directive: Explain
The core of a good answer
Fisher's linear discriminant is Y = (μ⁽¹⁾ − μ⁽²⁾)′Σ⁻¹x = (58x₁ + 48x₂)/47 ≈ 1.234x₁ + 1.021x₂; an individual is assigned to population 1 if Y ≥ ½(μ⁽¹⁾ − μ⁽²⁾)′Σ⁻¹(μ⁽¹⁾ + μ⁽²⁾) = 1746/47 ≈ 37.15 (equivalently 58x₁ + 48x₂ ≥ 1746), otherwise to population 2.
An outline written by upsc.help, not an official UPSC answer; every fact checked by a separate review. How we work
What the question asks
Part one is conceptual: assign an individual to one of two (or more) populations on the basis of measurements, with as few misclassifications as possible. Part two: define Fisher's function (the linear combination that maximises the ratio of between-group separation to within-group variance), then compute Σ⁻¹, the coefficient vector and the cut-off.
Answer outline
Introduction
Two populations π₁: N₂(μ⁽¹⁾, Σ) and π₂: N₂(μ⁽²⁾, Σ) with common known Σ; equal priors and costs assumed.
The classification problem
- An observation x of unknown origin is to be allotted to π₁ or π₂; the sample space is split into regions R₁ and R₂.
- Two errors: x from π₁ put in R₂, and vice versa; a good rule minimises total (expected) misclassification probability or cost.
- For normal populations with equal Σ, the optimal rule is linear in x.
Fisher's discriminant function
- Choose a to maximise [a′(μ⁽¹⁾ − μ⁽²⁾)]²/(a′Σa); the solution is a ∝ Σ⁻¹(μ⁽¹⁾ − μ⁽²⁾).
- Y = a′x; classify to π₁ if a′x ≥ ½a′(μ⁽¹⁾ + μ⁽²⁾).
Step 1: Inverse of Σ
- |Σ| = 9×8 − 25 = 47; Σ⁻¹ = (1/47)[8 5; 5 9].
Step 2: Coefficients
- δ = μ⁽¹⁾ − μ⁽²⁾ = (6, 2)′.
- a = Σ⁻¹δ = (1/47)(48 + 10, 30 + 18)′ = (58/47, 48/47)′ = (1.234, 1.021)′.
Step 3: Cut-off
- μ⁽¹⁾ + μ⁽²⁾ = (42, 22)′; ½a′(μ⁽¹⁾ + μ⁽²⁾) = (58×42 + 48×22)/94 = 3492/94 = 1746/47 = 37.15.
- Check: a′μ⁽¹⁾ = 41.87 and a′μ⁽²⁾ = 32.43, and 37.15 is their midpoint.
Step 4: Separation
- Mahalanobis distance Δ² = δ′Σ⁻¹δ = 444/47 = 9.45; probability of misclassification each way = Φ(−Δ/2) = Φ(−1.54) ≈ 0.062.
Conclusion
Fisher's function is Y = 1.234x₁ + 1.021x₂; classify to population 1 when Y ≥ 37.15, otherwise to population 2. All values were computed exactly in Python.
Examples and data to cite
- R. A. Fisher developed the original two-group discriminant analysis in 1936.
Facts to remember
- Fisher's criterion: maximise (a′δ)²/(a′Σa) ⇒ a = Σ⁻¹δ.
- Rule: assign to π₁ if (μ⁽¹⁾ − μ⁽²⁾)′Σ⁻¹x ≥ ½(μ⁽¹⁾ − μ⁽²⁾)′Σ⁻¹(μ⁽¹⁾ + μ⁽²⁾).
- Inverse of a 2 × 2 matrix [a b; c d] is (1/(ad − bc))[d −b; −c a].
- Mahalanobis Δ² = δ′Σ⁻¹δ; each error probability = Φ(−Δ/2) under equal priors.