Week 4: Lasso and Variable Selection

This week extends penalized regression from continuous shrinkage to sparse fitted models. The lasso can set some fitted coefficients exactly to zero, but a zero or nonzero coefficient must be interpreted in the context of the full predictor set, the selected penalty, and correlation among predictors.

Guiding question: How can a penalty perform variable selection, and when should we be cautious about the selected variables?

Learning goals

By the end of this week, you should be able to:

  • write the lasso objective for centered and standardized data;
  • explain exact zeros through the soft-thresholding rule;
  • explain how signal strength and the penalty affect selection across repeated samples;
  • connect coordinate descent to the lasso objective;
  • choose the penalty by cross-validation without using the final test data; and
  • interpret selected variables cautiously when predictors are correlated.

Lecture notes

  1. Lasso and Sparsity

    The absolute-value penalty, soft thresholding, repeated selection, coordinate descent, cross-validation, and the limits of variable selection under correlation.

Homework 04

Homework 04 contains four required problems. Questions 1 and 2 study variable selection under strong correlation using the lasso and the elastic net, Question 3 implements coordinate descent to compute a complete lasso path, and Question 4 tunes and compares lasso, ridge, and elastic net by cross-validation on the diabetes data. Use either R or Python. All homework assignments are evaluated as Complete or Incomplete. Attempt each problem before consulting the solutions.

Note: Due to the delay in publishing Homework 04, all students will receive full score for this homework regardless of the submission status.

ImportantDeadline

Place homework-04.pdf or homework-04.html in the homework/week-04/ folder of your stat432-fall2026 GitHub repository by Sunday, September 20, 2026, at 11:59 p.m. Central Time. The report must be either a PDF or a self-contained HTML file and must not exceed 10 pages. The final file version shown on GitHub is treated as the submitted version, and its GitHub timestamp determines whether it is on time.

Discussion Session on Tuesday, September 22, 2026

ImportantSubmit Discussion Question

Submit discussion/week-04-question.md by Sunday, September 20, 2026, at 11:59 p.m. Central Time. This required weekly discussion question counts toward the Weekly GitHub Discussion Questions component, worth 10% of the course grade.

The final version shown on GitHub is the submitted version. If its GitHub timestamp is after the deadline, the submission receives 0 points. No late-submission exceptions are allowed.

For the required format and GitHub submission steps, see Discussion Questions: Procedures and Rules.

NoteDiscussion session

The Week 4 discussion session is Tuesday, September 22, 2026. Open Popping when the instructor asks you to join.

See the Discussion tab for login information. Tell the instructor during class if you cannot sign in. There is no quiz during this session.

Quiz 2 is Tuesday, October 6, 2026. It covers the Week 3 through Week 5 discussions, including Week 4. See Quizzes for the quiz format.

Discussion Question Topics

Your question may focus on one of the following topics:

  1. Penalty geometry and sparsity. Why the lasso can produce exact zeros while ridge regression usually cannot.

  2. Soft thresholding. How the one-variable lasso solution changes as the penalty increases and why a range of values is mapped to zero.

  3. Penalty strength and prediction. How the choice of λ\lambda changes bias, variance, sparsity, and prediction error.

  4. Standardization and the intercept. Why predictor scaling matters and why the intercept is ordinarily left unpenalized.

  5. Choosing the penalty. Why cross-validation must be completed without using the final test data.

  6. Correlated predictors. Why different samples or nearby penalty values may select different variables even when prediction is stable.

NoteChoosing a question

Ask one focused question that calls for an explanation, a comparison, a short derivation, a counterexample, or a small computation. Do not submit a request for code alone or repeat a homework calculation without adding a statistical question.

Week 4 Check List