How to get w and b in SVR? (package e1071)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to get w and b in SVR? (package e1071)

marlene marchena-2
Dear R users,

I'm running a SVR in package e1071 but I did not able to calculate the
parameters w and b of the regression. I don't know how to do that and if it
is possible to do it with this package.

Someone have some idea. Any help would be much appreciated.

Marlene

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: How to get w and b in SVR? (package e1071)

Steve Lianoglou-6
Hi Marlene,

On Aug 3, 2009, at 8:59 AM, marlene marchena wrote:

> Dear R users,
>
> I'm running a SVR in package e1071 but I did not able to calculate the
> parameters w and b of the regression. I don't know how to do that  
> and if it
> is possible to do it with this package.
>
> Someone have some idea. Any help would be much appreciated.

While you're not given the W vector directly, you can recover it from  
the support vectors, no? In LaTeX form, W looks like:

  \vec{W} = \sum_{i=1}^N \alpha_i y_i \vec{x}_i

If "model" was the object returned from your SVM: alpha_i's are the  
weights of each vector (model$coefs[i]) and x_i are the support  
vectors (model$SV[i,]).

You can use some matrix multiplication to calculate this very quickly:

W <- t(model$coefs) %*% model$SV

By default, the x's and y's are scaled when you call svm(..) so I  
think, by definition, your y intercept (b) should be zero. I think the  
values in model$y.scale might be what you're looking for, though (not  
sure).

Hope that helps,
-steve

--
Steve Lianoglou
Graduate Student: Computational Systems Biology
   |  Memorial Sloan-Kettering Cancer Center
   |  Weill Medical College of Cornell University
Contact Info: http://cbio.mskcc.org/~lianos/contact

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: How to get w and b in SVR? (package e1071)

marlene marchena-2
Hi Steve,

First of all, thanks for your answer.

I did all that you suggested and now I have some questions.


1) when I use > svm.m1$coefs I have 180 values (the number of SV) of 10 and
-10, I think that it is related with C=10 because when I change for C=100 I
have 180 values of 100 and -100. I can't understand the meaning of that in
the SVR model. Do you know what it means?

2) For svm.m1$y.scale I get two values $`scaled:center` and $`scaled:scale.
The first one is the original value and the second one is the scale value
then my b will be scaled:center. Is that correct?

3) Which is the best form to put the data for SVM: in its original form
without scale and using scale=TRUE or using normalize date using
(x-min)/(max-min) like NN with scale=FALSE?

Thanks again,

Marlene.


Bellow my model:


> svm.m1  (I chose the parameters of the model  using tune.svm() )



Call:

svm(formula = QQ ~ ., data = train, cost = 10, gamma = 1e-06)





Parameters:

   SVM-Type:  eps-regression

 SVM-Kernel:  radial

       cost:  10

      gamma:  1e-06

    epsilon:  0.1





Number of Support Vectors:  180



> str(svm.m1)

List of 30

 $ call     : language svm(formula = QQ ~ ., data = train, cost = 10, gamma
= 1e-06)

 $ type     : num 3

 $ kernel   : num 2

 $ cost     : num 10

 $ degree   : num 3

 $ gamma    : num 1e-06

 $ coef0    : num 0

 $ nu       : num 0.5

 $ epsilon  : num 0.1

 $ sparse   : logi FALSE

 $ scaled   : logi [1:7] TRUE TRUE TRUE TRUE TRUE TRUE ...

 $ x.scale  :List of 2

  ..$ scaled:center: Named num [1:7] 0.23 0.23 0.23 0.234 0.238 ...

  .. ..- attr(*, "names")= chr [1:7] "diff1" "diff2" "diff3" "diff4" ...

  ..$ scaled:scale : Named num [1:7] 0.183 0.182 0.182 0.187 0.194 ...

  .. ..- attr(*, "names")= chr [1:7] "diff1" "diff2" "diff3" "diff4" ...

 $ y.scale  :List of 2

  ..$ scaled:center: num 0.227

  ..$ scaled:scale : num 0.182

 $ nclasses : int 2

 $ levels   : num [1:120] -0.98398 0.00101 -0.63924 0.18159 1.72474 ...

 $ tot.nSV  : int 180

 $ nSV      : int [1:2] 0 0

 $ labels   : int [1:2] 0 0

 $ SV       : num [1:180, 1:7] 1.4296 -0.9948 -0.012 -0.6508 0.1682 ...

  ..- attr(*, "dimnames")=List of 2

  .. ..$ : chr [1:180] "1" "2" "3" "4" ...

  .. ..$ : chr [1:7] "diff1" "diff2" "diff3" "diff4" ...

 $ index    : int [1:180] 1 2 3 4 5 6 7 8 9 10 ...

 $ rho      : num 0.139

 $ compprob : logi FALSE

 $ probA    : NULL

 $ probB    : NULL

 $ sigma    : NULL

 $ coefs    : num [1:180, 1] -10 10 -10 10 10 -10 -10 10 10 10 ...

 $ na.action: NULL

 $ fitted   : Named num [1:196] 0.202 0.203 0.202 0.202 0.202 ...

  ..- attr(*, "names")= chr [1:196] "1" "2" "3" "4" ...

 $ residuals: Named num [1:196] -1.1864 -0.2018 -0.8417 -0.0206 1.5224 ...

  ..- attr(*, "names")= chr [1:196] "1" "2" "3" "4" ...

 $ terms    :Classes 'terms', 'formula' length 3 QQ ~ diff1 + diff2 + diff3
+ diff4 + diff5 + diff6 + diff7

  .. ..- attr(*, "variables")= language list(QQ, diff1, diff2, diff3, diff4,
diff5, diff6, diff7)

  .. ..- attr(*, "factors")= int [1:8, 1:7] 0 1 0 0 0 0 0 0 0 0 ...

  .. .. ..- attr(*, "dimnames")=List of 2

  .. .. .. ..$ : chr [1:8] "QQ" "diff1" "diff2" "diff3" ...

  .. .. .. ..$ : chr [1:7] "diff1" "diff2" "diff3" "diff4" ...

  .. ..- attr(*, "term.labels")= chr [1:7] "diff1" "diff2" "diff3" "diff4"
...

  .. ..- attr(*, "order")= int [1:7] 1 1 1 1 1 1 1

  .. ..- attr(*, "intercept")= num 0

  .. ..- attr(*, "response")= int 1

  .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv>

  .. ..- attr(*, "predvars")= language list(QQ, diff1, diff2, diff3, diff4,
diff5, diff6, diff7)

  .. ..- attr(*, "dataClasses")= Named chr [1:8] "numeric" "numeric"
"numeric" "numeric" ...

  .. .. ..- attr(*, "names")= chr [1:8] "QQ" "diff1" "diff2" "diff3" ...

 - attr(*, "class")= chr [1:2] "svm.formula" "svm"



> svm.m1$coefs (some examples, I have 180 SV)

       [,1]

  [1,]  -10

  [2,]   10

  [3,]  -10

  [4,]   10

  [5,]   10

  [6,]  -10

  [7,]  -10

  [8,]   10

  [9,]   10

 [10,]   10



> svm.m1$SV

           diff1               diff2        diff3        diff4
diff5
diff6        diff7

1    1.429563809 -0.539452422 -0.376635036  3.227053246  3.765529896
0.072436459
-0.898142838

2   -0.994802571  1.429804286 -0.540826469 -0.386515451  3.087548981
3.780174219  0.087268726

3   -0.011951338 -0.998945652  1.429470704 -0.546407873 -0.394807530
3.100422296  3.821459922

4   -0.650804641 -0.014317301 -1.000562474  1.372301171 -0.548894103
-0.391030760  3.136858202

5    0.168238056 -0.654325731 -0.015413890 -0.994106648  1.300144754
-0.545519835 -0.379505172

6    1.708038328  0.166197899 -0.655760471 -0.034752129 -0.980336501
1.308349045
-0.535096474

7   -0.585281225  1.708782321  0.165196685 -0.658332568 -0.055817075
-0.978089239  1.331999124

8   -0.683566350 -0.588683840  1.708596139  0.141129535 -0.656754703
-0.051154801 -0.970752111

9   -0.028332192 -0.687146676 -0.590083898  1.644118288  0.113678155
-0.653662187 -0.037204315

10   0.659663673 -0.030727774 -0.688598758 -0.594375599  1.562091927
0.118783181
-0.644010384



> W = t(svm.m1$coefs) %*% svm.m1$SV

> W

         diff1    diff2    diff3     diff4    diff5    diff6     diff7

[1,] -10.81136 60.39054 7.716997 -62.35804 92.76012 222.9277 -65.45207



model$y.scale



> svm.m1$y.scale

$`scaled:center`

[1] 0.2273616



$`scaled:scale`

[1] 0.1823783


2009/8/3 Steve Lianoglou <[hidden email]>

> Hi Marlene,
>
>
> On Aug 3, 2009, at 8:59 AM, marlene marchena wrote:
>
>  Dear R users,
>>
>> I'm running a SVR in package e1071 but I did not able to calculate the
>> parameters w and b of the regression. I don't know how to do that and if
>> it
>> is possible to do it with this package.
>>
>> Someone have some idea. Any help would be much appreciated.
>>
>
> While you're not given the W vector directly, you can recover it from the
> support vectors, no? In LaTeX form, W looks like:
>
>  \vec{W} = \sum_{i=1}^N \alpha_i y_i \vec{x}_i
>
> If "model" was the object returned from your SVM: alpha_i's are the weights
> of each vector (model$coefs[i]) and x_i are the support vectors
> (model$SV[i,]).
>
> You can use some matrix multiplication to calculate this very quickly:
>
> W <- t(model$coefs) %*% model$SV
>
> By default, the x's and y's are scaled when you call svm(..) so I think, by
> definition, your y intercept (b) should be zero. I think the values in
> model$y.scale might be what you're looking for, though (not sure).
>
> Hope that helps,
> -steve
>
> --
> Steve Lianoglou
> Graduate Student: Computational Systems Biology
>  |  Memorial Sloan-Kettering Cancer Center
>  |  Weill Medical College of Cornell University
> Contact Info: http://cbio.mskcc.org/~lianos/contact<http://cbio.mskcc.org/%7Elianos/contact>
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.