Week 2: Training and Testing Errors

Weekly Objectives

By the end of this week, you should be able to:

• Be familiar with various components involved in a linear regression training and testing error decomposition. This includes the hat matrix, projection, random errors, rank of design matrix, etc.
• Derive and explain the expected training error \((n-p)\sigma^2\) and expected testing (prediction) error \((n+p)\sigma^2\) under a correctly specified linear model, and being able to challenge these results when different scenario arises.
• Explain and experiment how the testing error can be optimized in practice and provide counter examples on when such procedure could fail.
• Being able to implement these ideas via simulation studies when working on the homework questions or your self-generated examples.

Lecture Materials

• Lecture Notes: Bias-Variance Trade-off and Prediction Error

Homework Practice Questions

Practice questions will be posted here.