Intelligent Systems II · TEE · papers

Q1(d) Irreducibility and stationary distribution [5]

Re-examination · 25 Jun 2024

Actual historical paper ·100 marks attempted /140 offered · Q1 compulsory + any four of Q2–Q7. Source defects and solution assumptions are explicitly identified.

Question. Q1(d) [5 marks · 2024-06-25]

For a two-state Markov chain with P=[[0.8,0.2],[0.3,0.7]], (a) is the chain irreducible? (b) find its stationary distribution.

Answer

Given

Two states, row-stochastic matrix [[.8,.2],[.3,.7]].

To find

Mutual reachability and a probability row vector π with πP=π.

Formula and why it applies

Irreducible means every state can reach every other with positive probability in some number of steps. Stationary means one transition leaves the distribution unchanged.

LaTeX: \pi=\pi P,\quad\pi_1+\pi_2=1

Solution

Step 1. Check both directions

Swipe across the diagram to read its labels ↔

Diagram: Redrawn transition diagram. Each arrow goes from the current state to the next state; labels are probabilities.. Labels: legacy24r-stationary | 0.8 | 0.2 | 0.3 | 0.7 | 1 | 2

1→2 has probability .2>0 and 2→1 has probability .3>0. Each state reaches the other in one step; the chain is irreducible. Self-loops also make it aperiodic.

Step 2. Balance probability entering state 1

LaTeX: \pi_1=0.8\pi_1+0.3\pi_2\Rightarrow0.2\pi_1=0.3\pi_2\Rightarrow\pi_1=1.5\pi_2

Step 3. Normalize

LaTeX: 1.5\pi_2+\pi_2=1\Rightarrow\pi_2=0.4,\quad\pi_1=0.6

Step 4. Verify both columns

LaTeX: [0.6,0.4]P=[0.48+0.12,\;0.12+0.28]=[0.6,0.4]

Final answer. Yes, irreducible; π=[0.6,0.4].