Linear Regression and Bias-variance Trade-off

Week 2: Aug 28 – Sep 1

---

Weekly Objectives

• Be able to fit linear regression using R
• Understand the optimization problem involved in a linear regression
• Specify higher-order terms, categorical variables and interactions
• Use a linear model to predict future testing samples
• First try for understanding the bias-variance trade-off in prediction
• Be able to use model selection criteria and algorithms

---

Lecture Notes and R Examples

• Linear Regression [Source File]
• Model Selection [Source File]
• Basic concepts for Optimization [Source File]
• Optimization for Linear Regression [Source File]
• Automatic Differentiation with pytorch[Python Source File]
• Real Estate Data

Additional Readings

• Marrows’ \(C_p\)
• More about optimization

---

Homework

• Homework 2 [Source File]
• Due Sep 7, 11:59PM CT
• Solution