Markov Decision Process and Bellman Equation

---

Topics and Objectives (Apr 8 – Apr 12)

• Data generating process of Markov Decision Process
• Infinite horizon and discounted reward
• Bellman Equation and the optimal policy
• Contraction mapping theorem
• Bellman optimality equation
• Value iteration algorithm

---

Lecture Notes

• MDP and Bellman Equation
• Contractive Mapping and Value Iteration

---

Homework

• No new homework this week.
• See Homework 4 for details

---