Kernel Methods, Smoothing and Adaptiveness

Week 6: Sep 25 – Sep 29

Weekly Objectives

Previously we introduced KNN, which is a type of local averaging approach. In this week, we will introduce a general formulation for local estimations, defined using the concept of kernel functions. To improve smoothness, we introduce local polynomial. To address the curse of dimensional, we discuss a regression tree models, which builds adaptive kernels.

• Understand the Nadaraya-Watson kernel regression
• kernel weights and their similarities/differences with \(k\)NN
• Difference choices of kernel functions
• Tune \(h\) by understanding the bias-variance trade-off
• Local polynomial regressions
• Use the locpoly() function in KernSmooth package
• Regression and classification trees as adaptive kernel

Lecture Notes and R Examples

• Kernel Regression [Source File]
• Classification and Regression Trees [Source File]

Additional Readings

• Kernel Density Estimation

Homework

• Homework 6 [Source File]
• Due Oct 5, 11:59PM CT
• Solution