Chapter 9 Support Vector Machines, q8

In [3]:
library(ISLR)
OJ = na.omit(OJ)
summary(OJ)
 Purchase WeekofPurchase     StoreID        PriceCH         PriceMM     
 CH:653   Min.   :227.0   Min.   :1.00   Min.   :1.690   Min.   :1.690  
 MM:417   1st Qu.:240.0   1st Qu.:2.00   1st Qu.:1.790   1st Qu.:1.990  
          Median :257.0   Median :3.00   Median :1.860   Median :2.090  
          Mean   :254.4   Mean   :3.96   Mean   :1.867   Mean   :2.085  
          3rd Qu.:268.0   3rd Qu.:7.00   3rd Qu.:1.990   3rd Qu.:2.180  
          Max.   :278.0   Max.   :7.00   Max.   :2.090   Max.   :2.290  
     DiscCH            DiscMM         SpecialCH        SpecialMM     
 Min.   :0.00000   Min.   :0.0000   Min.   :0.0000   Min.   :0.0000  
 1st Qu.:0.00000   1st Qu.:0.0000   1st Qu.:0.0000   1st Qu.:0.0000  
 Median :0.00000   Median :0.0000   Median :0.0000   Median :0.0000  
 Mean   :0.05186   Mean   :0.1234   Mean   :0.1477   Mean   :0.1617  
 3rd Qu.:0.00000   3rd Qu.:0.2300   3rd Qu.:0.0000   3rd Qu.:0.0000  
 Max.   :0.50000   Max.   :0.8000   Max.   :1.0000   Max.   :1.0000  
    LoyalCH          SalePriceMM     SalePriceCH      PriceDiff       Store7   
 Min.   :0.000011   Min.   :1.190   Min.   :1.390   Min.   :-0.6700   No :714  
 1st Qu.:0.325257   1st Qu.:1.690   1st Qu.:1.750   1st Qu.: 0.0000   Yes:356  
 Median :0.600000   Median :2.090   Median :1.860   Median : 0.2300            
 Mean   :0.565782   Mean   :1.962   Mean   :1.816   Mean   : 0.1465            
 3rd Qu.:0.850873   3rd Qu.:2.130   3rd Qu.:1.890   3rd Qu.: 0.3200            
 Max.   :0.999947   Max.   :2.290   Max.   :2.090   Max.   : 0.6400            
   PctDiscMM        PctDiscCH       ListPriceDiff       STORE      
 Min.   :0.0000   Min.   :0.00000   Min.   :0.000   Min.   :0.000  
 1st Qu.:0.0000   1st Qu.:0.00000   1st Qu.:0.140   1st Qu.:0.000  
 Median :0.0000   Median :0.00000   Median :0.240   Median :2.000  
 Mean   :0.0593   Mean   :0.02731   Mean   :0.218   Mean   :1.631  
 3rd Qu.:0.1127   3rd Qu.:0.00000   3rd Qu.:0.300   3rd Qu.:3.000  
 Max.   :0.4020   Max.   :0.25269   Max.   :0.440   Max.   :4.000  

(a)

In [12]:
set.seed(1)
train = sample(nrow(OJ),800)

(b)

In [27]:
library(e1071)
set.seed(1)
svm.model = svm(Purchase~.,data=OJ[train,],kernel="linear",cost=0.01)
summary(svm.model)
Call:
svm(formula = Purchase ~ ., data = OJ[train, ], kernel = "linear", 
    cost = 0.01)


Parameters:
   SVM-Type:  C-classification 
 SVM-Kernel:  linear 
       cost:  0.01 
      gamma:  0.05555556 

Number of Support Vectors:  432

 ( 215 217 )


Number of Classes:  2 

Levels: 
 CH MM


The svm function uses radial kernel and produces 432 support vectors out of which 215 are from CH class and 217 from MM

(c)

In [19]:
#Training error
pred.train = predict(svm.model,newdata=OJ[train,])
mean(pred.train!=OJ[train,]$Purchase)*100
16.625
In [21]:
#Test error
pred.test = predict(svm.model,newdata=OJ[-train,])
mean(pred.test!=OJ[-train,]$Purchase)*100
18.1481481481481

(d)

In [59]:
set.seed(1)
range = list(cost=c(0.01,0.1,1,5,10))
tune.linear = tune(svm,Purchase~.,data=OJ[train,],kernel="linear",ranges=range)
summary(tune.linear)
Parameter tuning of ‘svm’:

- sampling method: 10-fold cross validation 

- best parameters:
 cost
 0.01

- best performance: 0.175 

- Detailed performance results:
   cost   error dispersion
1  0.01 0.17500 0.03996526
2  0.10 0.17875 0.03821086
3  1.00 0.17750 0.03717451
4  5.00 0.17875 0.03537988
5 10.00 0.18000 0.04005205

The cost of 0.01 gives the best result

(e)

In [60]:
#Training error with cost = 0.01
pred.train = predict(tune.linear$best.model,newdata=OJ[train,])
mean(pred.train!=OJ[train,]$Purchase)*100
table(pred.train,OJ[train,]$Purchase)
16.625
          
pred.train  CH  MM
        CH 439  78
        MM  55 228
In [61]:
#Test error with cost = 0.01
pred.test = predict(tune.linear$best.model,newdata=OJ[-train,])
mean(pred.test!=OJ[-train,]$Purchase)*100
table(pred.test,OJ[-train,]$Purchase)
18.1481481481481
         
pred.test  CH  MM
       CH 141  31
       MM  18  80

(f)

In [28]:
set.seed(2)
svm.radial = svm(Purchase~.,data=OJ[train,],kernel="radial",cost=0.01)
summary(svm.radial)
Call:
svm(formula = Purchase ~ ., data = OJ[train, ], kernel = "radial", 
    cost = 0.01)


Parameters:
   SVM-Type:  C-classification 
 SVM-Kernel:  radial 
       cost:  0.01 
      gamma:  0.05555556 

Number of Support Vectors:  617

 ( 306 311 )


Number of Classes:  2 

Levels: 
 CH MM


In [30]:
#Training error
pred.radial = predict(svm.radial,newdata=OJ[train,])
mean(pred.radial!=OJ[train,]$Purchase)*100
38.25
In [32]:
#Test error
pred.radial = predict(svm.radial,newdata=OJ[-train,])
mean(pred.radial!=OJ[-train,]$Purchase)*100
41.1111111111111
In [45]:
#cross validation
set.seed(1)
range = list(cost=c(0.01,0.1,1,5,10))
tune.radial = tune(svm,Purchase~.,data=OJ[train,],kernel="radial",ranges=range)
summary(tune.radial)
Parameter tuning of ‘svm’:

- sampling method: 10-fold cross validation 

- best parameters:
 cost
    5

- best performance: 0.17375 

- Detailed performance results:
   cost   error dispersion
1  0.01 0.38250 0.05596378
2  0.10 0.17875 0.04168749
3  1.00 0.17500 0.04750731
4  5.00 0.17375 0.04875178
5 10.00 0.18250 0.04866267
In [46]:
#Training error
pred.train = predict(tune.radial$best.model,newdata=OJ[train,])
mean(pred.train!=OJ[train,]$Purchase)*100
table(pred.train,OJ[train,]$Purchase)
13.75
          
pred.train  CH  MM
        CH 451  67
        MM  43 239
In [47]:
#Test error
pred.test = predict(tune.radial$best.model,newdata=OJ[-train,])
mean(pred.test!=OJ[-train,]$Purchase)*100
table(pred.test,OJ[-train,]$Purchase)
18.1481481481481
         
pred.test  CH  MM
       CH 140  30
       MM  19  81

(g)

In [49]:
set.seed(1)
svm.poly = svm(Purchase~.,data=OJ[train,],kernel="polynomial",degree=2,cost=0.01)
summary(svm.poly)
Call:
svm(formula = Purchase ~ ., data = OJ[train, ], kernel = "polynomial", 
    degree = 2, cost = 0.01)


Parameters:
   SVM-Type:  C-classification 
 SVM-Kernel:  polynomial 
       cost:  0.01 
     degree:  2 
      gamma:  0.05555556 
     coef.0:  0 

Number of Support Vectors:  620

 ( 306 314 )


Number of Classes:  2 

Levels: 
 CH MM


In [54]:
#Training error
pred.poly = predict(svm.poly,newdata=OJ[train,])
mean(pred.poly!=OJ[train,]$Purchase)*100
table(pred.poly,OJ[train,]$Purchase)
38.25
         
pred.poly  CH  MM
       CH 494 306
       MM   0   0
In [55]:
#Test error
pred.poly = predict(svm.poly,newdata=OJ[-train,])
mean(pred.poly!=OJ[-train,]$Purchase)*100
table(pred.poly,OJ[-train,]$Purchase)
41.1111111111111
         
pred.poly  CH  MM
       CH 159 111
       MM   0   0
In [56]:
#cross validation
range = list(cost=c(0.01,0.1,1,5,10))
tune.poly = tune(svm,Purchase~.,data=OJ[train,],kernel="polynomial",degree=2,ranges=range)
summary(tune.poly)
Parameter tuning of ‘svm’:

- sampling method: 10-fold cross validation 

- best parameters:
 cost
    5

- best performance: 0.1725 

- Detailed performance results:
   cost   error dispersion
1  0.01 0.38250 0.05596378
2  0.10 0.32375 0.06303934
3  1.00 0.19125 0.04860913
4  5.00 0.17250 0.05737305
5 10.00 0.17875 0.05653477
In [57]:
#Training error
pred.train = predict(tune.poly$best.model,newdata=OJ[train,])
mean(pred.train!=OJ[train,]$Purchase)*100
table(pred.train,OJ[train,]$Purchase)
14.875
         
pred.poly  CH  MM
       CH 454  79
       MM  40 227
In [58]:
#Test error
pred.test = predict(tune.poly$best.model,newdata=OJ[-train,])
mean(pred.test!=OJ[-train,]$Purchase)*100
table(pred.test,OJ[-train,]$Purchase)
18.1481481481481
         
pred.test  CH  MM
       CH 142  32
       MM  17  79

(h)

The training error and the test error in all the models are not significantly different.