# Q6(b) Markov process and homework transition paths [10]

Subject: Intelligent Systems II

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/is2/#tee/papers/paper-2024-reexam/q6b

Canonical: https://deep-learning-study-room.vercel.app/read/is2/tee/papers/paper-2024-reexam--q6b/

Actual historical paper ·100 marks attempted /140 offered · Q1 compulsory + any four of Q2–Q7. Source defects and solution assumptions are explicitly identified.

**Question. Q6(b)** [10 marks · 2024-06-25]

(I) Define the Markov process and list applications. (II) A professor assigns full-credit F, half-credit H and no-credit N homework according to the transition diagram. (a) Construct the TPM; (b) find P(X₃=F|X₂=N); (c) P(X₂=N|X₁=H); (d) P(X₄=H|X₂=F).

Swipe across the diagram to read its labels ↔

![Redrawn transition diagram. Each arrow goes from the current state to the next state; labels are probabilities.](https://deep-learning-study-room.vercel.app/is2/fig/legacy24r-homework.4001dfd62e.svg)

Diagram: Redrawn transition diagram. Each arrow goes from the current state to the next state; labels are probabilities.. Labels: legacy24r-homework | 0.3 | 0.25 | 0.45 | 0.45 | 0.4 | 0.15 | 0.65 | 0.25 | 0.1 | Full F | Half H | None N

Redrawn transition diagram. Each arrow goes from the current state to the next state; labels are probabilities.

### Answer

#### Given

The labeled arrows in the original diagram; state orderF, H, N.

#### To find

The transition matrix and one-step/two-step probabilities.

#### Formula and why it applies

A Markov process predicts the next state using the current state, conditionally independent of earlier history. Applications include weather regimes, brand switching, reliability, queueing and movement among locations. Here a state records the homework-credit category.



$$
P(X_{t+n}=j\mid X_t=i)=(P^n)_{ij}
$$



#### Solution

##### Step 1. Read all outgoing arrows into rows



$$
P=\begin{bmatrix}0.3&0.25&0.45\\0.45&0.4&0.15\\0.65&0.25&0.1\end{bmatrix}
$$



| Row | Sum |
| --- | --- |
| F | .30+.25+.45=1 |
| H | .45+.40+.15=1 |
| N | .65+.25+.10=1 |

##### Step 2. Part (b): one transition N→F



$$
P(X_3=F\mid X_2=N)=P_{NF}=0.65
$$



##### Step 3. Part (c): one transition H→N



$$
P(X_2=N\mid X_1=H)=P_{HN}=0.15
$$



##### Step 4. Part (d): two transitions F→?→H

The time difference is 4−2=2. Sum three mutually exclusive intermediate-state routes.

| Intermediate state | Path product |
| --- | --- |
| F | .30×.25=.075 |
| H | .25×.40=.100 |
| N | .45×.25=.1125 |



$$
(P^2)_{FH}=.075+.100+.1125=.2875
$$



**Final answer.** (b).65; (c).15; (d).2875; TPM shown above.