Given a transition probability matrix with three states {1, 2, 3} as :
P =
(0·6 0·3 0·1)
(0·2 0·6 0·2)
(0·4 0·4 0·2)
Find the value of P(X₅ = 2 | X₃ = 3) and P(X₃ = 1 | X₀ = 3).
10 marks · directive: Find
The core of a good answer
By the Chapman-Kolmogorov equations P(X₅ = 2 | X₃ = 3) is the (3, 2) entry of P², which is 0·44, and P(X₃ = 1 | X₀ = 3) is the (3, 1) entry of P³, which is 0·392 (exactly 49/125).
An outline written by upsc.help, not an official UPSC answer; every fact checked by a separate review. How we work
What the question asks
For a time-homogeneous Markov chain the n-step transition probabilities are entries of Pⁿ. The first probability is a two-step transition (from time 3 to 5), the second a three-step transition (from 0 to 3). Show the matrix products row by row; only the third row is needed.
Answer outline
Introduction
States 1, 2, 3; pᵢⱼ = P(Xₙ₊₁ = j | Xₙ = i). Time homogeneity gives P(Xₘ₊ₙ = j | Xₘ = i) = pᵢⱼ⁽ⁿ⁾, the (i, j) entry of Pⁿ.
Step 1: P² (row 3 is all that is needed, but full matrix shown)
- Row 3 of P² = (0·4, 0·4, 0·2) P
- p₃₁⁽²⁾ = 0·4(0·6) + 0·4(0·2) + 0·2(0·4) = 0·24 + 0·08 + 0·08 = 0·40
- p₃₂⁽²⁾ = 0·4(0·3) + 0·4(0·6) + 0·2(0·4) = 0·12 + 0·24 + 0·08 = 0·44
- p₃₃⁽²⁾ = 0·4(0·1) + 0·4(0·2) + 0·2(0·2) = 0·04 + 0·08 + 0·04 = 0·16
- P² = [[0·46, 0·40, 0·14], [0·32, 0·50, 0·18], [0·40, 0·44, 0·16]]
Step 2: First probability
- P(X₅ = 2 | X₃ = 3) = p₃₂⁽²⁾ = 0·44.
Step 3: Three-step probability
- p₃₁⁽³⁾ = Σₖ p₃ₖ⁽²⁾ pₖ₁ = 0·40(0·6) + 0·44(0·2) + 0·16(0·4)
- = 0·240 + 0·088 + 0·064 = 0·392
- P³ = [[0·412, 0·434, 0·154], [0·364, 0·468, 0·168], [0·392, 0·448, 0·160]]
- P(X₃ = 1 | X₀ = 3) = 0·392.
Step 4: Check
- Each row of P² and P³ sums to 1 (for example 0·40 + 0·44 + 0·16 = 1 and 0·392 + 0·448 + 0·160 = 1).
- Exact fractions: 11/25 and 49/125 (computed in Python).
Conclusion
P(X₅ = 2 | X₃ = 3) = 0·44 and P(X₃ = 1 | X₀ = 3) = 0·392; the rows of the power matrices sum to one, confirming the arithmetic.
Examples and data to cite
- As n grows, every row of Pⁿ tends to the stationary distribution π solving π = πP for an irreducible aperiodic chain such as this one.
Facts to remember
- Markov property: the future depends on the past only through the present state.
- Chapman-Kolmogorov: pᵢⱼ⁽ᵐ⁺ⁿ⁾ = Σₖ pᵢₖ⁽ᵐ⁾ pₖⱼ⁽ⁿ⁾, so P⁽ⁿ⁾ = Pⁿ.
- Time homogeneity: P(Xₘ₊ₙ = j | Xₘ = i) does not depend on m.
- Rows of every power of a stochastic matrix sum to 1.