Lasso

Week 4: Sep 15 – Sep 19

Weekly Objectives

• The solution the Lasso penalty in the one-variable case as an optimization problem
• The effect of the Lasso penalty with the constrained formulation in a two-dimensional case
• The different properties of Ridge and Lasso and know when to use which
• Understand the connection and difference between Lasso, step-wise regression and stage-wise regression.
• Fit Lasso and Elastic-net using the glmnet package and extract useful information
• Use penalized logistic regression models for classification problems
• Understand and address some common practical issues such as missing values, outliers, highly skewed and non-standard distribution

Lecture Notes and R Examples

• Lasso and Variable Selection [Source File]
• Logistic and Penalized Logistic Regression [Source File]
• Practical Issues in Regression [Source File]

Additional Readings

• Path-wise Coordinate Descent for Lasso

Homework

• [Homework 4]
• Due Sep 25, 11:59PM CT
• [Solution]