Natural Language Processing · TEE · formulas

HMM path probabilities: three complete numerical examples

HMM & decoding

Formulas explained through full worked examples

The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table.

Initial state	Probability
H	0.8
C	0.2

From → to	H	C
H	0.7	0.3
C	0.4	0.6

Emission state	Observation 1	Observation 2	Observation 3
H	0.2	0.4	0.4
C	0.5	0.4	0.1

Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.

Given

Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor.

To find

P(H,C,H; 1,3,1).

Formula and why it applies

P=\pi_H b_H(1)a_{HC}b_C(3)a_{CH}b_H(1)

There are three hidden states and observations: three emissions, two transitions, and one initial probability.

Solution

Step 1. Start Hot and emit 1

0.8\times0.2=0.16

Step 2. Transition Hot to Cold

0.16\times0.3=0.048

Step 3. Cold emits 3

0.048\times0.1=0.0048

Step 4. Transition Cold to Hot

0.0048\times0.4=0.00192

Step 5. Hot emits the last 1

0.00192\times0.2=0.000384

Final answer. Joint probability = 0.000384 = 3.84×10⁻⁴.

Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.

Given

State path H→H→H, observations 1→3→1; no END factor.

To find

P(H,H,H;1,3,1).

Formula and why it applies

P=\pi_H b_H(1)a_{HH}b_H(3)a_{HH}b_H(1)

The state remains Hot; each transition still contributes its probability 0.7.

Solution

Step 1. Initial state and first emission

0.8(0.2)=0.16

Step 2. First self-transition

0.16(0.7)=0.112

Step 3. Emit observation 3

0.112(0.4)=0.0448

Step 4. Second self-transition

0.0448(0.7)=0.03136

Step 5. Emit final observation 1

0.03136(0.2)=0.006272

Final answer. Joint probability = 0.006272. A self-transition is not automatically probability 1.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.