R › R help

mgcv bam() with grouped binomial data

Classic

List

Threaded

6 messages Options

Casey Olives

mgcv bam() with grouped binomial data

Hello,

I'm trying to use the bam() function in the R mgcv package for a large set of grouped binary data. However, I have found that this function does not take data in the format of cbind(numerator, denominator) on the left hand side of the formula. As an example, consider the following

dat1 <- data.frame(id=rep(1:6, each=3), num=rbinom(18, size=10, prob=0.8), den=rbinom(18, size=5, prob=0.5), x=rnorm(18))
m1.1 <- gam(cbind(num, den) ~ x+s(id, bs="re"), data=dat1, family=binomial)
m2.1 <- bam(cbind(num, den) ~x+ s(id, bs="re"), data=dat1, family=binomial)

Running the above results in

> dat1 <- data.frame(id=rep(1:6, each=3), num=rbinom(18, size=10, prob=0.8), den=rbinom(18, size=5, prob=0.5), x=rnorm(18))
> m1.1 <- gam(cbind(num, den) ~ x+s(id, bs="re"), data=dat1, family=binomial)
> m2.1 <- bam(cbind(num, den) ~x+ s(id, bs="re"), data=dat1, family=binomial)
Error in dev.resids(y, mu, weights) :
argument mu must be a numeric vector of length 1 or length 36

Is this a known issue? If so, is there a way to handle it?

Thank you,

Casey

[[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Simon Wood-4

Re: mgcv bam() with grouped binomial data

upgrade to the current mgcv version (see last item in changeLog for mgcv
1.7-12)

On 01/30/2012 10:04 PM, Casey Olives wrote:

> Hello,
>
> I'm trying to use the bam() function in the R mgcv package for a large set of grouped binary data. However, I have found that this function does not take data in the format of cbind(numerator, denominator) on the left hand side of the formula. As an example, consider the following
>
> dat1<- data.frame(id=rep(1:6, each=3), num=rbinom(18, size=10, prob=0.8), den=rbinom(18, size=5, prob=0.5), x=rnorm(18))
> m1.1<- gam(cbind(num, den) ~ x+s(id, bs="re"), data=dat1, family=binomial)
> m2.1<- bam(cbind(num, den) ~x+ s(id, bs="re"), data=dat1, family=binomial)
>
> Running the above results in
>
>> dat1<- data.frame(id=rep(1:6, each=3), num=rbinom(18, size=10, prob=0.8), den=rbinom(18, size=5, prob=0.5), x=rnorm(18))
>> m1.1<- gam(cbind(num, den) ~ x+s(id, bs="re"), data=dat1, family=binomial)
>> m2.1<- bam(cbind(num, den) ~x+ s(id, bs="re"), data=dat1, family=binomial)
> Error in dev.resids(y, mu, weights) :
> argument mu must be a numeric vector of length 1 or length 36
>
> Is this a known issue? If so, is there a way to handle it?
>
> Thank you,
>
> Casey
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Casey Olives

Re: mgcv bam() with grouped binomial data

Hi Simon,

Thanks for your response. I am now running mgcv 1.7-13 on R-2.14. Having fixed this error, I now encounter the following when fitting this model

Error in qr.qty(qrx, yn) : 'qr' and 'y' must have the same number of rows
Calls: bam -> bgam.fit -> qr.update -> qr.qty -> qr.qty

bam(cbind(group0, temp)~1+ as.factor(AGEcat) + HSMF+BachMF+income.cpi+poverty+tobacco+tax+CigSalesPerCap+blacks+hisp+asian+native +s( year.centered , by= fips , bs='cr', k=8) +s( fips , bs='re'), gamma=1.6, family=binomial(link="logit"), data=data

My sense is that the error is coming from the term

s( year.centered , by= fips , bs='cr', k=8)

as if I run the same model with

s( year.centered, bs='cr', k=8)

I don't get the error. Thus, something about trying to fit these smooths by fips id is causing the error.

Do you have any ideas on what might be happening? I recognize it is hard to know without seeing the data....

Thanks again
Casey

-----Original Message-----
From: Simon Wood [mailto:[hidden email]]
Sent: Tuesday, January 31, 2012 2:17 AM
To: Casey Olives
Cc: '[hidden email]'
Subject: Re: [R] mgcv bam() with grouped binomial data

upgrade to the current mgcv version (see last item in changeLog for mgcv
1.7-12)

On 01/30/2012 10:04 PM, Casey Olives wrote:

Simon Wood-4

Re: mgcv bam() with grouped binomial data

Hi Casey,

any chance that you could send me the data off list (under a strict
undertaking to not use it for anything but this de-bugging of course)? I
can't immediately reproduce the problem by simulating similar, but it
certainly looks like a bug.

best,
Simon

On 02/02/2012 01:54 AM, Casey Olives wrote:

> Hi Simon,
>
> Thanks for your response. I am now running mgcv 1.7-13 on R-2.14. Having fixed this error, I now encounter the following when fitting this model
>
> Error in qr.qty(qrx, yn) : 'qr' and 'y' must have the same number of rows
> Calls: bam -> bgam.fit -> qr.update -> qr.qty -> qr.qty
>
> bam(cbind(group0, temp)~1+ as.factor(AGEcat) + HSMF+BachMF+income.cpi+poverty+tobacco+tax+CigSalesPerCap+blacks+hisp+asian+native +s( year.centered , by= fips , bs='cr', k=8) +s( fips , bs='re'), gamma=1.6, family=binomial(link="logit"), data=data
>
> My sense is that the error is coming from the term
>
> s( year.centered , by= fips , bs='cr', k=8)
>
> as if I run the same model with
>
> s( year.centered, bs='cr', k=8)
>
> I don't get the error. Thus, something about trying to fit these smooths by fips id is causing the error.
>
> Do you have any ideas on what might be happening? I recognize it is hard to know without seeing the data....
>
> Thanks again
> Casey
>
> -----Original Message-----
> From: Simon Wood [mailto:[hidden email]]
> Sent: Tuesday, January 31, 2012 2:17 AM
> To: Casey Olives
> Cc: '[hidden email]'
> Subject: Re: [R] mgcv bam() with grouped binomial data
>
> upgrade to the current mgcv version (see last item in changeLog for mgcv
> 1.7-12)
>
> On 01/30/2012 10:04 PM, Casey Olives wrote:
>> Hello,
>>
>> I'm trying to use the bam() function in the R mgcv package for a large set of grouped binary data. However, I have found that this function does not take data in the format of cbind(numerator, denominator) on the left hand side of the formula. As an example, consider the following
>>
>> dat1<- data.frame(id=rep(1:6, each=3), num=rbinom(18, size=10, prob=0.8), den=rbinom(18, size=5, prob=0.5), x=rnorm(18))
>> m1.1<- gam(cbind(num, den) ~ x+s(id, bs="re"), data=dat1, family=binomial)
>> m2.1<- bam(cbind(num, den) ~x+ s(id, bs="re"), data=dat1, family=binomial)
>>
>> Running the above results in
>>
>>> dat1<- data.frame(id=rep(1:6, each=3), num=rbinom(18, size=10, prob=0.8), den=rbinom(18, size=5, prob=0.5), x=rnorm(18))
>>> m1.1<- gam(cbind(num, den) ~ x+s(id, bs="re"), data=dat1, family=binomial)
>>> m2.1<- bam(cbind(num, den) ~x+ s(id, bs="re"), data=dat1, family=binomial)
>> Error in dev.resids(y, mu, weights) :
>> argument mu must be a numeric vector of length 1 or length 36
>>
>> Is this a known issue? If so, is there a way to handle it?
>>
>> Thank you,
>>
>> Casey
>>
>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

ywh123

Re: mgcv bam() with grouped binomial data

This post has NOT been accepted by the mailing list yet.

Hi,Dear Professor Wood,
I am studying the generalized additive model(GAM)
and I have read your related papers,especially the book named "Generalized
Additive Models:An Introduction with R".I am benefit a lots.However,I also
have some questions about GAM models.

First,Is there some restrictions on the sample size? For example,I am studying
the GDP and foreign direct investment on 29 provinces in China(N=29).Whether or
not N is too samll? If so,could I use pooled data(N=29,T=5)?

Second,Could I use the "mgcv" packages to implement panel data model through adding
specific fixed effect and time fixed effect as the generzlied mixed model?(eg. adding factor(s)).

Thanks very much in advance.

Best Regards,
Wanhai

ywh123

Re: mgcv bam() with grouped binomial data

This post has NOT been accepted by the mailing list yet.

In reply to this post by Simon Wood-4

Hi,Dear Professor Wood, I am studying the generalized additive model(GAM) and I have read your related papers,especially the book named "Generalized Additive Models:An Introduction with R".I am benefit a lots.However,I also have some questions about GAM models. First,Is there some restrictions on the sample size? For example,I am studying the GDP and foreign direct investment on 29 provinces in China(N=29).Whether or not N is too samll? If so,could I use pooled data(N=29,T=5)? Second,Could I use the "mgcv" packages to implement panel data model through adding specific fixed effect and time fixed effect as the generzlied mixed model?(eg. adding factor(s)). Thanks very much in advance. Best Regards, Wanhai