Linear Regression and Bias-variance Trade-off

Week 2: Sep 2 – Sep 6

---

Weekly Objectives

• Be able to fit linear regression using R
• Understand the optimization problem involved in a linear regression
• Specify higher-order terms, categorical variables and interactions
• Use a linear model to predict future testing samples
• First try for understanding the bias-variance trade-off in prediction
• Use simulation to demonstrate properties of a model
• Be able to use model selection criteria and algorithms

---

Lecture Notes and R Examples

• Linear Regression [Source File]
• Model Selection [Source File]
• Basic concepts for Optimization [Source File]
• Optimization for Linear Regression [Source File]
• Automatic Differentiation with pytorch[Python Source File]

Data

• Real Estate Data

Additional Readings

• Marrows’ \(C_p\)
• More about optimization

---

Homework

• Homework 2 [Source File]
• Due Sep 12, 11:59PM CT
• Solution