GLM with regularization

[Dmitriy Lyubimov]

Hello,

Thank you for probably not so new question, but i am new to R.

Does any of packages have something like glm+regularization? So far i
see probably something close to that as a ridge regression in MASS but
I think i need something like GLM, in particular binomial regularized
versions of polynomial regression.

Also I am not sure how some of the K-fold crossvalidation helpers out
there (cv.glm) could be used to adjust reg rate as there seems to be
no way to apply them over data not used for training (or  i am not
seeing a solution here as training is completely separated from
crossvalidation error computation here) .

The example here in cv.glm doesn't look right to me since it computes
cv error over model trained on 100% of data. (e.g. wikipedia
crossvalidation article lists this as an example of misuse of K-fold
CV).


----- doc quote ----
# leave-one-out and 6-fold cross-validation prediction error for
# the mammals data set.
data(mammals, package="MASS")
mammals.glm <- glm(log(brain)~log(body),data=mammals)
cv.err <- cv.glm(mammals,mammals.glm)
cv.err.6 <- cv.glm(mammals, mammals.glm, K=6)
---- end of quote ---


Those seem to be pretty common techniques, any poniter in the right
direction (package) will be greatly appreciated.

thank you very much.
-Dmitriy

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Bert Gunter]

Google is your friend! -- as usual.

If you had searched on "glm with regularization" you would have bumped
into the glmnet R package, which I think is what you're looking for.

-- Bert

On Wed, Feb 29, 2012 at 6:22 PM, Dmitriy Lyubimov <[hidden email]> wrote:

> Hello,
>
> Thank you for probably not so new question, but i am new to R.
>
> Does any of packages have something like glm+regularization? So far i
> see probably something close to that as a ridge regression in MASS but
> I think i need something like GLM, in particular binomial regularized
> versions of polynomial regression.
>
> Also I am not sure how some of the K-fold crossvalidation helpers out
> there (cv.glm) could be used to adjust reg rate as there seems to be
> no way to apply them over data not used for training (or  i am not
> seeing a solution here as training is completely separated from
> crossvalidation error computation here) .
>
> The example here in cv.glm doesn't look right to me since it computes
> cv error over model trained on 100% of data. (e.g. wikipedia
> crossvalidation article lists this as an example of misuse of K-fold
> CV).
>
>
> ----- doc quote ----
> # leave-one-out and 6-fold cross-validation prediction error for
> # the mammals data set.
> data(mammals, package="MASS")
> mammals.glm <- glm(log(brain)~log(body),data=mammals)
> cv.err <- cv.glm(mammals,mammals.glm)
> cv.err.6 <- cv.glm(mammals, mammals.glm, K=6)
> ---- end of quote ---
>
>
> Those seem to be pretty common techniques, any poniter in the right
> direction (package) will be greatly appreciated.
>
> thank you very much.
> -Dmitriy
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--

Bert Gunter
Genentech Nonclinical Biostatistics

Internal Contact Info:
Phone: [phone]
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Dmitriy Lyubimov]

Thank you.

On Thu, Mar 1, 2012 at 9:58 AM, Bert Gunter <[hidden email]> wrote:

> Google is your friend! -- as usual.
>
> If you had searched on "glm with regularization" you would have bumped
> into the glmnet R package, which I think is what you're looking for.
>
> -- Bert
>
> On Wed, Feb 29, 2012 at 6:22 PM, Dmitriy Lyubimov <[hidden email]> wrote:
>> Hello,
>>
>> Thank you for probably not so new question, but i am new to R.
>>
>> Does any of packages have something like glm+regularization? So far i
>> see probably something close to that as a ridge regression in MASS but
>> I think i need something like GLM, in particular binomial regularized
>> versions of polynomial regression.
>>
>> Also I am not sure how some of the K-fold crossvalidation helpers out
>> there (cv.glm) could be used to adjust reg rate as there seems to be
>> no way to apply them over data not used for training (or  i am not
>> seeing a solution here as training is completely separated from
>> crossvalidation error computation here) .
>>
>> The example here in cv.glm doesn't look right to me since it computes
>> cv error over model trained on 100% of data. (e.g. wikipedia
>> crossvalidation article lists this as an example of misuse of K-fold
>> CV).
>>
>>
>> ----- doc quote ----
>> # leave-one-out and 6-fold cross-validation prediction error for
>> # the mammals data set.
>> data(mammals, package="MASS")
>> mammals.glm <- glm(log(brain)~log(body),data=mammals)
>> cv.err <- cv.glm(mammals,mammals.glm)
>> cv.err.6 <- cv.glm(mammals, mammals.glm, K=6)
>> ---- end of quote ---
>>
>>
>> Those seem to be pretty common techniques, any poniter in the right
>> direction (package) will be greatly appreciated.
>>
>> thank you very much.
>> -Dmitriy
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>
>
> --
>
> Bert Gunter
> Genentech Nonclinical Biostatistics
>
> Internal Contact Info:
> Phone: [phone]
> Website:
> http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.