caret: compare linear models of different degree

[Dominik Bruhn]

I want to use the caret package to train linear models. I want to
compare these models when using different degrees (aka degrees of
interaction). This is possible for the 'earth' method (using the
'.degree' parameter) but I found no possibility of customizing the
degree for the 'lm' method. This might be due to the fact that the basic
'lm' function does not support the degree parameter.

I tried the following solution using a different formula for the second
training:
-----
library(caret)
data(trees)
formula=Volume~Girth+Height
degFormula=Volume~(Girth+Height)^2
m1 = train(formula, data=trees, method="lm")
m2 = train(degFormula, data=trees, method="lm")
-------

The problem with this solution is that the 'extractPrediction' method of
R does not work if comparing models with different formulas:

---
Error in eval(expr, envir, enclos) : object 'Girth:Height' not found
---

How can I set the degree used when training a linear model without
changing the formula? I need the 'extractPrediction' functionallity.

Thanks!
--
Dominik Bruhn
mailto: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Dominik Bruhn]

Max, thanks for your answer!
> predict.train() will handle the formulas. If you want to compare the
> models in terms of their predictive performance, set the seeds prior
> to running the model. This will ensure that the same resampling
> indices are used in train(). If you do this, the resamples() function
> can be used to make formal comparisons between the models:

I think I did not express my question in the right way: I want to use
the extractPrediction function because I want to plot some stats using
the plotObsVsPred method. As stated in my previous mail, the
extractPrediction method does not work if the formula is changed.

Perhaps my question now got clearer.

Thanks again!

--
Dominik Bruhn
mailto: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Dominik Bruhn]

The same statement is true for the "plotmo" function. It also does not
handle the situations right if the training functions contains
interactions. You can try this out using this code:

----
library(caret)
data(trees)
m = train(Volume~(Girth+Height)^2, data=trees, method="lm")
plotmo(m$finalModel)
---

which leads to this error:
----
Error in eval(expr, envir, enclos) : object 'Girth:Height' not found
----

Is this a bug or am I missing something?

Thanks!

On 09/06/12 18:09, Dominik Bruhn wrote:

> Max, thanks for your answer!
>> predict.train() will handle the formulas. If you want to compare the
>> models in terms of their predictive performance, set the seeds prior
>> to running the model. This will ensure that the same resampling
>> indices are used in train(). If you do this, the resamples() function
>> can be used to make formal comparisons between the models:
>
> I think I did not express my question in the right way: I want to use
> the extractPrediction function because I want to plot some stats using
> the plotObsVsPred method. As stated in my previous mail, the
> extractPrediction method does not work if the formula is changed.
>
> Perhaps my question now got clearer.
>
> Thanks again!
>

--
Dominik Bruhn
mailto: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.