Syllabus

---

Course Overview

This course aims to cover selected advanced statistical modeling tools beyond those in Stat 432 and Stat 542. This is the first time that I am offering the full version of the course, and I plan to cover these main topics:

• Functional approximations and Reproducing Kernel Hilbert Space (RKHS) methods
• Advanced view of random forests and associated statistical tools
• Personalized medicine, especially in the context of health care
• Reinforcement learning, fundamental concepts and algorithms

Especially for the last two topics, they will utilize the tools and methods covered in the first two topics. There is no required textbook for this course. However, we will provide lecture notes and homework assignments in R Markdown. You can use either R or Python to complete your homework assignments. Detailed tentative topics and objectives are summarized in the following.

---

Reproducing Kernel Hilbert Space

Topics covered in this section include:

• Functional approximation basics
• Constructing Reproducing kernel Hilbert space (RKHS) and Properties
• The kernel trick and computations of kernel methods
• Representer theorem and its applications
• Mercer’s Theorem and Moore–Aronszajn Theorem
• Kernel ridge regression
• Support vector machines and regression
• Kernel PCA
• Kernel mean embedding

Some references:

• Hofmann, T., Schölkopf, B., & Smola, A. J. (2005). A tutorial review of RKHS methods in machine learning. Technical Report.
• Gretton, A. (2013). Introduction to rkhs, and some simple kernel algorithms. Adv. Top. Mach. Learn. Lecture Conducted from University College London, 16(5-3), 2.

---

Random Forests

Topics covered in this section include:

• Random forests and kernel view
• Generalized random forests via influence function split
• Resampling approaches
• Uncertainty quantification with U-statistics
• Variable importance and its variations
• Boosting and other ensemble methods

Some references

• Geurts, P., Ernst, D., & Wehenkel, L. (2006). Extremely randomized trees. Machine learning, 63(1), 3-42.
• Scornet, E. (2016). Random forests and kernel methods. IEEE Transactions on Information Theory, 62(3), 1485-1500.
• Athey, S., Tibshirani, J., and Wager, S. (2019) .Generalized random forests. The Annals of Statistics, 47(2), 1148-1178.
• Wager, S., Hastie, T., & Efron, B. (2014). Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife. Journal of Machine Learning Research, 15, 1625-1651.
• Mentch, L., & Hooker, G. (2016). Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests. Journal of Machine Learning Research, 17, 1-41.
• Ishwaran, H., & Lu, M. (2019). Standard errors and confidence intervals for variable importance in random forest regression, classification, and survival. Statistics in medicine, 38(4), 558-582.
• Cui, Y., Kosorok, M. R., Sverdrup, E., Wager, S., & Zhu, R. (2023). Estimating heterogeneous treatment effects with right-censored data via causal survival forests. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(2), 179-211.
• Xu, T., Zhu, R., & Shao, X. (2024). On variance estimation of random forests with Infinite-order U-statistics. Electronic Journal of Statistics, 18(1).

---

Personalized Medicine and Reinforcement Learning

Topics covered in this section include:

• Randomized trials, observational study and propensity score weighting
• Treatment effect estimations and unobserved confounders
• Estimating the optimal treatment regime
• Dynamic treatment rules and reinforcement learning
• Bellman equation, its properties and applications
• Example algorithms for policy evaluation and policy learning
• Other topics in reinforcement learning TBD

Some references:

• For treatment effects estimation and casual inference, you can consider:
  • Imbens, G. W., & Rubin, D. B. (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.
  • A First Course in Causal Inference by Peng Ding
  • STATS 361: Casual Inference by Stefan Wager
  • Applied Causal Inference Powered by ML and AI by V. Chernozhukov, C. Hansen, N. Kallus, M. Spindler and V. Syrgkanis
• For reinforcement learning
  • The goto book is Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press.
  • Reinforcement learning: Theory and algorithms by Alekh Agarwal, Nan Jiang, Sham Kakade and Wen Sun
  • Algorithms of Reinforcement Learning by Csaba Szepesvari

---

Prerequisites

You need to be familiar with basic statistical theory, linear algebra, linear regression and some common machine learning models. We assume that you are able to utilize any models covered in STAT 432, at least computationally, to implement our introduced methods.

---

Software

You need to be familiar with basic programming in R and Python. Lecture notes and homework assignments will be provided in R Markdown and/or Jupyter Notebook. You can use either R and Python to complete your homework assignments. However, you are responsible for installing the required packages and libraries.

---

Using AI Tools

As we explore programming and machine learning, I encourage you to use AI tools in your homework to gain insights, automate routine tasks, and enrich your learning. These tools can be powerful aids, but they have limitations. Relying on them without a solid grasp of the underlying principles can lead to errors and misconceptions. Always be mindful of biases in data and the ethical implications of AI use. You must also report any use of AI tools in each homework submission and final project. Treat these tools as extensions of your skills—never as substitutes for critical thinking and rigorous understanding. Used responsibly, they can significantly enhance your educational experience.

The above paragraph was generated with the help of GPT-5. The university also has a discussion about ChatGPT. There is also a Guidance for Students. When writing your homework and final project, I would also consider this statement by Elsevier. If you have questions or concerns, please let me know or talk to an expert.

---

Gradings

The current plan is to have six to eight sets of homework assignments, but may not be equally weighted. Each homework may contain 50-200 points. Some homework assignments contain a mini-project. Letter grades are based on your total points.

A+	A	A-	B+	B	B-	C+	C	C-	D+	D	D-
TBD	93%	90%	87%	83%	80%	77%	73%	70%	67%	63%	60%

---

Academic Integrity

The official University of Illinois policy related to academic integrity can be found in Article 1, Part 4 of the Student Code. Section 1-402 in particular outlines behavior which is considered an infraction of academic integrity. These sections of the Student Code will be upheld in this course. Any violations will be dealt with in a swift, fair and strict manner. Homework assignments are meant to be learning experiences. You may discuss the exercises with other students, but you must write up the solutions on your own. In short, do not cheat, it is not worth the risk. You are more likely to get caught than you believe. If you think you may be operating in a gray area, you most likely are.

---

Changes

The instructor reserves the right to make any changes he considers academically advisable. Such changes, if any, will be announced on this website and through email. Please note that it is your responsibility to keep track of the proceedings.

---