Please remove this section when submitting your homework.
Students are encouraged to work together on homework and/or utilize advanced AI tools. However, sharing, copying, or providing any part of a homework solution or code to others is an infraction of the University’s rules on Academic Integrity. Any violation will be punished as severely as possible. Final submissions must be uploaded to Gradescope. No email or hard copy will be accepted. For late submission policy and grading rubrics, please refer to the course website.
HWx_yourNetID.pdf
. For example,
HW01_rqzhu.pdf
. Please note that this must be a
.pdf
file. .html
format
cannot be accepted. Make all of your R
code chunks visible for grading..Rmd
file
as a template, be sure to remove this instruction
section.R
is \(\geq
4.0.0\). This will ensure your random seed generation is the same
as everyone else. Please note that updating the R
version
may require you to reinstall all of your packages.During our lecture, we considered a simulation model to analyze the variable selection property of Lasso. Now let’s further investigate the prediction error cased by the \(L1\) penalty under this model, and understand the bias-variance trade-off. For this question, your underlying true data generate should be
\[\begin{align} Y =& X^\text{T} \boldsymbol \beta + \epsilon \\ =& \sum_{j = 1}^p X_j 0.4^\sqrt{j} + \epsilon, \end{align}\]
where p
\(= 30\), each
\(X_j\) is generated independently from
\(\cal{N}(0, 1)\), and \(\epsilon\) also follows a standard normal,
independent from \(X\). The goal is to
predict two target points and investigate how the prediction error
changes under different penalties. The training data and two target
testing points are defined by the following code.
# target testing points
p = 30
xa = xb = rep(0, p)
xa[2] = 1
xb[10] = 1
Perform the following questions:
exp(seq(-5, 5, 0.05))
nsim
\(= 200\) independent runs, with
n
\(= 100\) observations in
each run.
glmnet()
function to fit Lasso on the \(\lambda\) valuesxb
should be much larger than xa
. What are the
corresponding best \(\lambda\) and
Error for each target point?xb
is much
larger than xa
. Hint: pay attention to their covariate
values and the associated \(\widehat
\beta\) parameters. Discuss how their predictions would trade
bias and variance differently.In HW3, we used golub
dataset from the
multtest
package. This dataset contains 3051 genes from 38
tumor mRNA samples from the leukemia microarray study Golub et
al. (1999). The outcome golub.cl
is an indicator for two
leukemia types: Acute Lymphoblastic Leukemia (ALL) or Acute Myeloid
Leukemia (AML). In genetic analysis, many gene expressions are highly
correlated. Hence we could consider the Elastic net model for both
sparsity and correlation.