Ensemble Models: Random Forests and Boosting

Week 10: Oct 23 – Oct 27

Weekly Objectives

• Understand the basic structure of fitting trees and random forests
• Splitting rules in both classification and regression settings
• Know how to tune mtry, nodesize and other parameters using the randomForest package
• Know the effect of these different tuning on the performance, especially the impact on the bias-variance trade-off.
• Understand the kernel view of random forests
• Understand the Adaboost algorithm
• Compare linear boosting with Lasso
• Compare boosting with random forests

Lecture Notes and R Examples

• Random Forests [Source File]
• Boosting [Source File]

Additional Readings

• AdaBoost Weight Update
• Gradient Boosting

Homework

• Homework 9 [Source File]
• Due Nov 2, 11:59PM CT
• Solution