Reinforcement Learning

---

Topics and Objectives

• MDP and RL formulation
• Bellman equation
• Contraction Mapping Theorem
• Value Iteration
• Model-free vs. Model-based RL
  
• Fitted Q-Iteration
• Function approximation in RL
• Online vs. Offline RL
• Algorithms for solving MDPs

---

Week 12 (Nov 11 and Nov 13)

• Nov 4: [Markov Decision Processes]
• Nov 6: [Contraction Mapping Theorem]

---

Week 13 (Nov 18 and Nov 20)

• Nov 6: [Fitted Q-Iteration] [CartPole Simulator]
• Nov 8: [Policy Gradient]

---

Homework 6

• [Homework 6] [source code]
• Processed data can be found here (see folder Expedia Data (Condensed MDP)). There is a training set avalilable, which is sufficient for the homework. However you can utlize the testing data for validation if you want.
• [Ice Lake Figure]
• Due Dec 10, 11:59PM CT

---

Week 14 Fall Break

---

Week 15 (Nov 18 and Nov 20)

• Dec 2: [Policy Iteration and SAC]