# Unit 4 · MRP, MDP, policies and Bellman equations

Subject: Intelligent Systems II

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/is2/#tee/read/u4-bellman

Canonical: https://deep-learning-study-room.vercel.app/read/is2/tee/read-chapter/u4-bellman/

Unit 4 · Markov reward process, Markov decision process, value functions, Bellman equations, applications

1. [Markov reward process, return and discount](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u4-bellman--mrp-return/)
2. [Bellman equation for an MRP (with a solved example)](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u4-bellman--bellman-mrp/)
3. [Markov decision process, policies and value functions](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u4-bellman--mdp-policy/)
4. [Bellman expectation and optimality equations (worked)](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u4-bellman--bellman-equations/)
5. [Using Bellman equations: policy and value iteration](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u4-bellman--solving/)
6. [10-mark answer: Bellman equation and applications of RL](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u4-bellman--ten-mark-answer/)