Stat 432 Homework 7

Instruction

Please remove this section when submitting your homework.

Students are encouraged to work together on homework and/or utilize advanced AI tools. However, there are two basic rules:

• sharing, copying, or providing any part of a homework solution or code to others is an infraction of the University’s rules on Academic Integrity. Any violation will be punished as severely as possible. This simply means: write your own solution.
• You must indicate clearly the level of involvement of AI tools (ChatGPT, Copilot, etc.) at the beginning of your homework. This includes whether you used it to generate code, debug code, or write explanations. If you did not use any AI tools, please state that explicitly. You are encouraged to briefly document your prompt(s) especially when AI is heavily involved. Please be aware that you are responsible any consequences of using AI tools, including incorrect or misleading information.

Final submissions must be uploaded to Gradescope. No email or hard copy will be accepted. Please refer to the course website for late submission policy and grading rubrics.

• You are required to submit the rendered file HWx_yourNetID.pdf. For example, HW01_rqzhu.pdf. Please note that this must be a .pdf file. .html format will not be accepted because they are often not readable on gradescope. Make all of your R code chunks visible for grading.
• Include your Name and NetID at the beginning of your homework.
• Make sure that you set seed properly so that the results can be replicated if needed. Make sure the version of your R is \(\geq 4.0.0\). This will ensure your random seed generation is the same as everyone else.
• For some questions, there will be restrictions on what packages/functions you can use. Please read the requirements carefully. As long as the question does not specify such restrictions, you can use anything.
• If you use this file or the example homework .Rmd file as a template, be sure to remove this instruction section.

Question 1 [40 pts]: Linear SVM and support vectors

We will use the Social Network Ads data, available on Kaggle [link]. The .csv file is also available at our course website. The goal is to classify the outcome Purchased, and we will only use the two continuous variables EstimatedSalary and Age. Scale and center both covariates before you proceed with these following steps. For this question, you should use the e1071 package. Complete the following tasks:

• [5 pts] Randomly select (without replacement) 100 observations with Purchased == 1 and 100 with Purchased == 0 to form a training data set. The rest of the data will be used as the testing data. Set seed before you do this.
  
• [5 pts] Produce a 2-d scatter plot for your training data, with each observation colored by the outcome. Use pch = 19 for the dots.
• [5 pts] Fit a linear SVM with cost = 1. Since your data is already scaled and centered, set scale = FALSE in the svm() function.
• [5 pts] Extract the decision line from the fitted object, and draw the decision line on the plot. Also draw the two margin lines (\(f(x) = \pm 1\)) using dashed lines. For this question, you cannot use any build-in functions to draw the decision line. You need to calculate the slope and intercept of the decision line and margin lines yourself based on the quantities you obtained from the svm() fitted object.
• [5 pts] Mark the support vectors on the plot (with a circle on the observation, use cex = 2).
• [5 pts] What is the training data (in-sample) classification accuracy? Provide a confusion table of the results.
• [5 pts] Use your decision line to predict the outcome for the testing data. What is the testing data classification accuracy? Provide a confusion table of the results. For this question, you cannot use the predict() function. You need to calculate the predicted outcome based on the decision line you obtained.
• [5 pts] Since this is a linear SVM, we can actually use the fitted function \(f(x)\) as a linear predictor to calculate the ROC curve and AUC. Use the ROCR package to calculate the AUC and plot the ROC curve for the testing data. You need to calculate \(f(x)\) yourself based on the quantities you obtained from the fitted object.

Question 2 [30 pts]: Nonlinear SVM

In this question, we will use the same training and testing data from the previous question. Complete the following tasks. For this question, you can use the predict() function to make predictions.

• [5 pts] Fit a SVM with cubic kernel with cost = 1 and coef0 = 1. Since your data is already scaled and centered, set scale = FALSE in the svm() function.
• [5 pts] What is the training data (in-sample) classification accuracy? Provide a confusion table of the results.
• [5 pts] What is the testing data classification accuracy? Provide a confusion table of the results.
• [5 pts] Use the ROCR package to calculate the AUC and plot the ROC curve for the testing data. Is this better than the linear SVM?
• [10 pts] Generate a two-dimensional grid of values for EstimatedSalary and Age that cover the range of the data. Use the fitted SVM model to predict the outcome for each point in the grid. Plot these grid points with different colors for different predicted outcomes so that you can visualize the decision boundary.

Question 3 [30 pts]: SVM for hand written digit Data

Take digits 4 and 9 from zip.train and zip.test in the ElemStatLearn package. For this question, you should use the kernlab package, in combination with the caret package to tune the parameters. Make sure that you specify the method = argument so that the correct package/function is used to fit the model. You may consider reading the details from this documentation. Complete the following task.

• [5 Points] Construct the training and testing data (with digits 4 and 9) so that they become a binary classification problem.
• [10 Points] Construct a grid of tuning parameters \(C\) (cost) for linear SVM using the kernlab package, and tune this using caret. Use 10-fold cross-validation for this question. What is the best C you obtained based on the accuracy? Predict the testing data using this model and obtain the confusion table and testing data accuracy.
• [15 Points] Construct a grid of tuning parameters for radial Kernel SVM using the kernlab package, and tune this using caret. Use repeated 10-fold cross-validation for this question, and repeat 3 times. You may need to modify your code get a good range of tuning parameter. What is the best C and sigma you obtained based on the accuracy? Predict the testing data using this model and obtain the confusion table and testing data accuracy.