Ensemble Models: Random Forests and Boosting

Week 9: Oct 20 – Oct 24

Weekly Objectives

• Understand the basic structure of fitting trees and random forests
• Splitting rules in both classification and regression settings
• Know how to tune mtry, nodesize and other parameters using the randomForest package
• Know the effect of these different tuning on the performance, especially the impact on the bias-variance trade-off.
• Understand the kernel view of random forests
• Understand the Adaboost algorithm
• Compare linear boosting with Lasso
• Compare boosting with random forests

Lecture Notes and R Examples

• Random Forests [Source File]
• Boosting [Source File]

Additional Readings

• AdaBoost Weight Update
• Gradient Boosting

Homework

• [Homework 9]
• Due Oct 30, 11:59PM CT
• [Solution]