K-nearest neighbors and nonparametric estimation

Week 5: Sep 18 – Sep 22

Weekly Objectives

• Theoretical Understanding of bias-variance trade-off
• Understand the KNN algorithm
  • The effect of local averaging
  • The effect of \(k\) on the bias-variance trade-off
  • Tune \(k\) using the caret package
  • Various distance measures
  • Computational issue
• Curse of dimensionality
  • high-dimensional issue of local averaging methods
  • Use simulation studies to understand the problem
• Classification
  • Evaluate a classification model

Lecture Notes and R Examples

• K-Nearest Neighbors [Source File]
• The Curse of Dimensionality [Source File]
• Classification Model Evaluation [Source File]
• Dataset:
  • [Cleveland heart disease data]
  • Handwritten Digit Data from the ElemStatLearn package

Additional Readings

• Bias-variance Trade-off

Homework

• Homework 5 [Source File]
• Due Sep 28, 11:59PM CT
• Solution