# Q1(d) Irreducibility and stationary distribution [5]

Subject: Intelligent Systems II

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/is2/#tee/papers/paper-2024-reexam/q1d

Canonical: https://deep-learning-study-room.vercel.app/read/is2/tee/papers/paper-2024-reexam--q1d/

Actual historical paper ·100 marks attempted /140 offered · Q1 compulsory + any four of Q2–Q7. Source defects and solution assumptions are explicitly identified.

**Question. Q1(d)** [5 marks · 2024-06-25]

For a two-state Markov chain with P=[[0.8,0.2],[0.3,0.7]], (a) is the chain irreducible? (b) find its stationary distribution.

### Answer

#### Given

Two states, row-stochastic matrix [[.8,.2],[.3,.7]].

#### To find

Mutual reachability and a probability row vector π with πP=π.

#### Formula and why it applies

Irreducible means every state can reach every other with positive probability in some number of steps. Stationary means one transition leaves the distribution unchanged.



$$
\pi=\pi P,\quad\pi_1+\pi_2=1
$$



#### Solution

##### Step 1. Check both directions

Swipe across the diagram to read its labels ↔

![Redrawn transition diagram. Each arrow goes from the current state to the next state; labels are probabilities.](https://deep-learning-study-room.vercel.app/is2/fig/legacy24r-stationary.41b587e21f.svg)

Diagram: Redrawn transition diagram. Each arrow goes from the current state to the next state; labels are probabilities.. Labels: legacy24r-stationary | 0.8 | 0.2 | 0.3 | 0.7 | 1 | 2

Redrawn transition diagram. Each arrow goes from the current state to the next state; labels are probabilities.

1→2 has probability .2>0 and 2→1 has probability .3>0. Each state reaches the other in one step; the chain is irreducible. Self-loops also make it aperiodic.

##### Step 2. Balance probability entering state 1



$$
\pi_1=0.8\pi_1+0.3\pi_2\Rightarrow0.2\pi_1=0.3\pi_2\Rightarrow\pi_1=1.5\pi_2
$$



##### Step 3. Normalize



$$
1.5\pi_2+\pi_2=1\Rightarrow\pi_2=0.4,\quad\pi_1=0.6
$$



##### Step 4. Verify both columns



$$
[0.6,0.4]P=[0.48+0.12,\;0.12+0.28]=[0.6,0.4]
$$



**Final answer.** Yes, irreducible; π=[0.6,0.4].