Ridge Regression

Week 3: Sep 4 – Sep 8

Weekly Objectives

• Correlated variables and motivation of Ridge regression
• How to solve Ridge regression
• The source of bias and variance, and their trade-offs
• The effect of penalty and different ways of tuning it
• \(k\)-fold Cross-validation
• Scaling issue and other thing that could affect the performance
• Using the glmnet package and lm.ridge() from the MASS package

Lecture Notes and R Examples

• Ridge Regression and Bias-variance Trade-off [Source File]
• We will also briefly introduce PCA [Source File]

Additional Readings

• Derivation of the Bias-variance Trade-off in a simplified case
• Ridge regression scaling issue

Homework

• Homework 3 [Source File]
• Due Sep 14, 11:59PM CT
• Solution