Homework 06 Solutions

Development template

Homework 06 solutions are being prepared.

Download the materials

Question 1: Compare KNN and logistic regression

Original question

Use breast-cancer.csv, which contains 569 cases from the Wisconsin Diagnostic Breast Cancer dataset. Predict y (1 for malignant and 0 for benign) using mean_radius, mean_texture, and mean_smoothness.

Set a random seed and randomly split the observations into 70% training and 30% test data. Use the same split for every model. Standardize the predictors using the training means and standard deviations, and apply the same transformations to the test data.

Fit ordinary logistic regression with an intercept and no penalty, and KNN with Euclidean distance, equal neighbor weights, and

$k\in\{5,15,25,35,45\}.$

Using the estimated probabilities for y = 1 on the test data, plot all six ROC curves in one figure and report their AUC values. Which of the six fitted models do you prefer? Give a brief justification based on your results.

Solution

To be added.

Question 2: Training and test ROC curves

Original question

Use the same data, training/test split, and standardization as in Question 1. Fit KNN with Euclidean distance, equal neighbor weights, and

$k\in\{1,3,5,10,20\}.$

Use each fitted model to estimate probabilities for both the training and test observations. For training predictions, include each observation itself among its $k$ nearest neighbors.

For each $k$ , use a separate panel to show the training and test ROC curves together. Report both AUC values.

Explain the difference between training and test performance and how it changes with $k$ . In particular, explain what happens at $k=1$ and why training performance may not reflect performance on new observations.

Solution

To be added.

Question 3: To be added

Original question

To be added.

Solution

To be added.

Question 4

No solution can be provided for Question 4.

Question 5: To be added

Original question

To be added.

Solution

To be added.