unexpected "false convergence"

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

unexpected "false convergence"

Jack Tanner
I've come into some code that produces different results under R 2.1.1 and R
2.2.1. I'm really unfamiliar with the libraries in question (MASS and nlme),
so I don't know if this is a bug in my code, or a regression in R. If it's a
bug on my end, I'd appreciate any advice on potential causes and relevant
documentation.

The code:

score<-c(1,8,1,3,4,4,2,5,3,6,0,3,1,5,0,5,1,11,1,2,4,5,2,4,1,6,1,2,8,16,5,16,3,15,3,12,4,9,2,4,1,8,2,6,4,11,2,9,3,17,2,6)
id<-rep(1:13,rep(4,13))
test<-gl(2,1,52,labels=c("pre","post"))
coder<-gl(2,2,52,labels=c("two","three"))
il<-data.frame(id,score,test,coder)
attach(il)
cs1<-corSymm(value=c(.396,.786,.718,.639,.665,.849),form=~1|id)
cs1<-Initialize(cs1,data=il)
run<-glmmPQL(score~test+coder,
random=~1|id,family=poisson,data=il,correlation=cs1)

The output under R 2.2.1, which leaves the run object (last line of the
code) undefined:

iteration 1
iteration 2
iteration 3
iteration 4
Error in lme.formula(fixed = zz ~ test + coder, random = ~1 | id, data =
list( :
        false convergence (8)

Under R 2.1.1, I get exactly 4 iterations as well, but no "false
convergence" message, and run is defined.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: unexpected "false convergence"

Jack Tanner
I should've added that I looked in the changelogs for R, nlme, and MASS, and
didn't see anything obviously related, but, as I said, I'm really unfamiliar
with those libraries.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: unexpected "false convergence"

Spencer Graves
In reply to this post by Jack Tanner
          I replicated your 'false convergence' using R 2.2.0:
 > sessionInfo()
R version 2.2.0, 2005-10-06, i386-pc-mingw32

attached base packages:
[1] "methods"   "stats"     "graphics"  "grDevices" "utils"     "datasets"
[7] "base"

other attached packages:
     nlme     MASS
"3.1-66" "7.2-23"

          Since the error message said, "Error in lme.formula", I listed the
code for "lme.formula" and traced it using "debug(lme.formula)",  The
function "glmmPQL" calls "lme.formula" repeatedly.  The function
"lme.formula" in turn calls "nlminb" when it's available, though it used
to call "optim".  The fifth time "lme.formula" was called, "nlminb"
returned the error message "false convergence (8)".

          Under R 2.2, "nlminb" is part of the "base" package.  I'm not
certain, but I don't think it was available in "base" under R 2.1.1.

          I think this explains the problem, but not how to fix it.  I tried
modifying the code fo "lme.formula" to force it to call "optim", but
this generated a different error.  I am therefore copying Professors
Bates & Ripley in case one of them might want to look at this.

          hope this helps.
          spencer graves

Jack Tanner wrote:

> I've come into some code that produces different results under R 2.1.1 and R
> 2.2.1. I'm really unfamiliar with the libraries in question (MASS and nlme),
> so I don't know if this is a bug in my code, or a regression in R. If it's a
> bug on my end, I'd appreciate any advice on potential causes and relevant
> documentation.
>
> The code:
>
> score<-c(1,8,1,3,4,4,2,5,3,6,0,3,1,5,0,5,1,11,1,2,4,5,2,4,1,6,1,2,8,16,5,16,3,15,3,12,4,9,2,4,1,8,2,6,4,11,2,9,3,17,2,6)
> id<-rep(1:13,rep(4,13))
> test<-gl(2,1,52,labels=c("pre","post"))
> coder<-gl(2,2,52,labels=c("two","three"))
> il<-data.frame(id,score,test,coder)
> attach(il)
> cs1<-corSymm(value=c(.396,.786,.718,.639,.665,.849),form=~1|id)
> cs1<-Initialize(cs1,data=il)
> run<-glmmPQL(score~test+coder,
> random=~1|id,family=poisson,data=il,correlation=cs1)
>
> The output under R 2.2.1, which leaves the run object (last line of the
> code) undefined:
>
> iteration 1
> iteration 2
> iteration 3
> iteration 4
> Error in lme.formula(fixed = zz ~ test + coder, random = ~1 | id, data =
> list( :
>         false convergence (8)
>
> Under R 2.1.1, I get exactly 4 iterations as well, but no "false
> convergence" message, and run is defined.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: unexpected "false convergence"

Jack Tanner
Thank you, Spencer.

I've discovered that if I drop the value parameter from the call to
corSymm(), R 2.2.1 converges successfully, but the values it converges to
are slighly (in the hundredths and thousandths) different from those that R
2.1.1 produces.

In the same vein, I also have an example where the R 2.2.1 glmmPQL produces
the warning "non-integer #successes in a binomial glm! in: eval(expr, envir,
enclos)", and another where R 2.2.1 says "iteration limit reached without
convergence" unless I invoke glmmPQL() with lmeControl(msMaxIter=100).
Neither of those happen in R 2.1.1.

I'm fine with the results above. I don't know if they're of interest to
anyone else; if they are, I'm happy to provide more details.



>From: Spencer Graves <[hidden email]>
>To: Jack Tanner <[hidden email]>
>CC: [hidden email], Douglas Bates <[hidden email]>,        
>Prof Brian Ripley <[hidden email]>
>Subject: Re: [R] unexpected "false convergence"
>Date: Tue, 03 Jan 2006 20:40:56 -0800
>
>  I replicated your 'false convergence' using R 2.2.0:
> > sessionInfo()
>R version 2.2.0, 2005-10-06, i386-pc-mingw32
>
>attached base packages:
>[1] "methods"   "stats"     "graphics"  "grDevices" "utils"     "datasets"
>[7] "base"
>
>other attached packages:
>     nlme     MASS
>"3.1-66" "7.2-23"
>
>  Since the error message said, "Error in lme.formula", I listed the code
>for "lme.formula" and traced it using "debug(lme.formula)",  The function
>"glmmPQL" calls "lme.formula" repeatedly.  The function "lme.formula" in
>turn calls "nlminb" when it's available, though it used to call "optim".  
>The fifth time "lme.formula" was called, "nlminb" returned the error
>message "false convergence (8)".
>
>  Under R 2.2, "nlminb" is part of the "base" package.  I'm not certain,
>but I don't think it was available in "base" under R 2.1.1.
>
>  I think this explains the problem, but not how to fix it.  I tried
>modifying the code fo "lme.formula" to force it to call "optim", but this
>generated a different error.  I am therefore copying Professors Bates &
>Ripley in case one of them might want to look at this.
>
>  hope this helps.
>  spencer graves
>
>Jack Tanner wrote:
>>I've come into some code that produces different results under R 2.1.1 and
>>R 2.2.1. I'm really unfamiliar with the libraries in question (MASS and
>>nlme), so I don't know if this is a bug in my code, or a regression in R.
>>If it's a bug on my end, I'd appreciate any advice on potential causes and
>>relevant documentation.
>>
>>The code:
>>
>>score<-c(1,8,1,3,4,4,2,5,3,6,0,3,1,5,0,5,1,11,1,2,4,5,2,4,1,6,1,2,8,16,5,16,3,15,3,12,4,9,2,4,1,8,2,6,4,11,2,9,3,17,2,6)
>>id<-rep(1:13,rep(4,13))
>>test<-gl(2,1,52,labels=c("pre","post"))
>>coder<-gl(2,2,52,labels=c("two","three"))
>>il<-data.frame(id,score,test,coder)
>>attach(il)
>>cs1<-corSymm(value=c(.396,.786,.718,.639,.665,.849),form=~1|id)
>>cs1<-Initialize(cs1,data=il)
>>run<-glmmPQL(score~test+coder,
>>random=~1|id,family=poisson,data=il,correlation=cs1)
>>
>>The output under R 2.2.1, which leaves the run object (last line of the
>>code) undefined:
>>
>>iteration 1
>>iteration 2
>>iteration 3
>>iteration 4
>>Error in lme.formula(fixed = zz ~ test + coder, random = ~1 | id, data =
>>list( :
>>         false convergence (8)
>>
>>Under R 2.1.1, I get exactly 4 iterations as well, but no "false
>>convergence" message, and run is defined.
>>
>>______________________________________________
>>[hidden email] mailing list
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide!
>>http://www.R-project.org/posting-guide.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: unexpected "false convergence"

Prof Brian Ripley
nlme in 2.2.x uses a different optimizer (nlminb) from that used earlier
(optim). The author (Douglas Bates) believes it to be better, but my
experience is rather the opposite.

For some reason he choose not to give you the option via lmeControl of
selecting one or the other.  I also cannot find any announcement of the
change in the package itself (other than the ChangeLog).

On Wed, 4 Jan 2006, Jack Tanner wrote:

> Thank you, Spencer.
>
> I've discovered that if I drop the value parameter from the call to
> corSymm(), R 2.2.1 converges successfully, but the values it converges to
> are slighly (in the hundredths and thousandths) different from those that R
> 2.1.1 produces.
>
> In the same vein, I also have an example where the R 2.2.1 glmmPQL produces
> the warning "non-integer #successes in a binomial glm! in: eval(expr, envir,
> enclos)", and another where R 2.2.1 says "iteration limit reached without
> convergence" unless I invoke glmmPQL() with lmeControl(msMaxIter=100).
> Neither of those happen in R 2.1.1.
>
> I'm fine with the results above. I don't know if they're of interest to
> anyone else; if they are, I'm happy to provide more details.
>
>
>
>> From: Spencer Graves <[hidden email]>
>> To: Jack Tanner <[hidden email]>
>> CC: [hidden email], Douglas Bates <[hidden email]>,
>> Prof Brian Ripley <[hidden email]>
>> Subject: Re: [R] unexpected "false convergence"
>> Date: Tue, 03 Jan 2006 20:40:56 -0800
>>
>>  I replicated your 'false convergence' using R 2.2.0:
>>> sessionInfo()
>> R version 2.2.0, 2005-10-06, i386-pc-mingw32
>>
>> attached base packages:
>> [1] "methods"   "stats"     "graphics"  "grDevices" "utils"     "datasets"
>> [7] "base"
>>
>> other attached packages:
>>     nlme     MASS
>> "3.1-66" "7.2-23"
>>
>>  Since the error message said, "Error in lme.formula", I listed the code
>> for "lme.formula" and traced it using "debug(lme.formula)",  The function
>> "glmmPQL" calls "lme.formula" repeatedly.  The function "lme.formula" in
>> turn calls "nlminb" when it's available, though it used to call "optim".
>> The fifth time "lme.formula" was called, "nlminb" returned the error
>> message "false convergence (8)".
>>
>>  Under R 2.2, "nlminb" is part of the "base" package.  I'm not certain,
>> but I don't think it was available in "base" under R 2.1.1.
>>
>>  I think this explains the problem, but not how to fix it.  I tried
>> modifying the code fo "lme.formula" to force it to call "optim", but this
>> generated a different error.  I am therefore copying Professors Bates &
>> Ripley in case one of them might want to look at this.
>>
>>  hope this helps.
>>  spencer graves
>>
>> Jack Tanner wrote:
>>> I've come into some code that produces different results under R 2.1.1 and
>>> R 2.2.1. I'm really unfamiliar with the libraries in question (MASS and
>>> nlme), so I don't know if this is a bug in my code, or a regression in R.
>>> If it's a bug on my end, I'd appreciate any advice on potential causes and
>>> relevant documentation.
>>>
>>> The code:
>>>
>>> score<-c(1,8,1,3,4,4,2,5,3,6,0,3,1,5,0,5,1,11,1,2,4,5,2,4,1,6,1,2,8,16,5,16,3,15,3,12,4,9,2,4,1,8,2,6,4,11,2,9,3,17,2,6)
>>> id<-rep(1:13,rep(4,13))
>>> test<-gl(2,1,52,labels=c("pre","post"))
>>> coder<-gl(2,2,52,labels=c("two","three"))
>>> il<-data.frame(id,score,test,coder)
>>> attach(il)
>>> cs1<-corSymm(value=c(.396,.786,.718,.639,.665,.849),form=~1|id)
>>> cs1<-Initialize(cs1,data=il)
>>> run<-glmmPQL(score~test+coder,
>>> random=~1|id,family=poisson,data=il,correlation=cs1)
>>>
>>> The output under R 2.2.1, which leaves the run object (last line of the
>>> code) undefined:
>>>
>>> iteration 1
>>> iteration 2
>>> iteration 3
>>> iteration 4
>>> Error in lme.formula(fixed = zz ~ test + coder, random = ~1 | id, data =
>>> list( :
>>>         false convergence (8)
>>>
>>> Under R 2.1.1, I get exactly 4 iterations as well, but no "false
>>> convergence" message, and run is defined.
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide!
>>> http://www.R-project.org/posting-guide.html
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html