Bug in "stats4" package - "confint" method

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug in "stats4" package - "confint" method

Stefano de Pretis-3
Dear R developers,

I noticed a bug in the stats4 package, specifically in the confint method applied to “mle” objects.
In particular, when some “fixed” parameters define the log likelihood, these parameters are stored within the mle object but they are not used by the “confint" method, which retrieves their value from the global environment (whenever they still exist).

Sample code:

> ## Avoid printing to unwarranted accuracy
> od <- options(digits = 5)
> x <- 0:10
> y <- c(26, 17, 13, 12, 20, 5, 9, 8, 5, 4, 8)
>
> ## Easy one-dimensional MLE:
> nLL <- function(lambda, y) -sum(stats::dpois(y, lambda, log = TRUE))
> fit0 <- mle(nLL, start = list(lambda = 5), fixed=list(y=y), nobs = NROW(y))
> confint(fit0)
Profiling...
  2.5 %  97.5 %
 9.6524 13.6716
> rm(y)
> confint(fit0)
Profiling...
Error in eval(expr, p) : object 'y' not found

In this sample code, I’m showing that after the removal of y from the global environment the method ‘confint’ is not able anymore to act on the object ‘fit0’.

This retrieval from the global environment is actually dangerous because it’s silent and in case the user changes the value of ‘y’ for another purpose the method ‘confint’ will return a wrong evaluation of the confidence interval without throwing any warning.

I suggest to use the fixed values that are already contained within the mle object:

> fit0@fullcoef[grep('y', names(fit0@fullcoef))]
 y1  y2  y3  y4  y5  y6  y7  y8  y9 y10 y11
 26  17  13  12  20   5   9   8   5   4   8

Additionally, the method ‘coef’ returns both the coefficient and the fixed parameters - while should return only the parameters - , while the ‘fullcoef’ method is not implemented - while present in the documentation of 'mle-class’.

Best,
Stefano
 
Center for Genomic Science of IIT@SEMM

Stefano de Pretis, PhD
Postdoctoral fellow
[hidden email] <mailto:[hidden email]>
Via Adamello 16, 20139 Milan, Italy <http://maps.google.it/maps?q=Via+Adamello+16,+20139+Milan&oe=utf-8&client=firefox-a&ie=UTF8&hq=&hnear=Via+Adamello,+16,+20139+Milano,+Lombardia&gl=it&z=16>
t: +39 02 94375046



        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug in "stats4" package - "confint" method

Peter Dalgaard-2
Yep, I know. Well, maybe I didn't, but there are also issues with propagation of fixedness in constrained optimization.

Revising mle() had neen on my todo list for way too long. The bbmle package handles this better, I believe.

-pd

> On 24 Apr 2019, at 16:51 , Stefano de Pretis <[hidden email]> wrote:
>
> Dear R developers,
>
> I noticed a bug in the stats4 package, specifically in the confint method applied to “mle” objects.
> In particular, when some “fixed” parameters define the log likelihood, these parameters are stored within the mle object but they are not used by the “confint" method, which retrieves their value from the global environment (whenever they still exist).
>
> Sample code:
>
>> ## Avoid printing to unwarranted accuracy
>> od <- options(digits = 5)
>> x <- 0:10
>> y <- c(26, 17, 13, 12, 20, 5, 9, 8, 5, 4, 8)
>>
>> ## Easy one-dimensional MLE:
>> nLL <- function(lambda, y) -sum(stats::dpois(y, lambda, log = TRUE))
>> fit0 <- mle(nLL, start = list(lambda = 5), fixed=list(y=y), nobs = NROW(y))
>> confint(fit0)
> Profiling...
>  2.5 %  97.5 %
> 9.6524 13.6716
>> rm(y)
>> confint(fit0)
> Profiling...
> Error in eval(expr, p) : object 'y' not found
>
> In this sample code, I’m showing that after the removal of y from the global environment the method ‘confint’ is not able anymore to act on the object ‘fit0’.
>
> This retrieval from the global environment is actually dangerous because it’s silent and in case the user changes the value of ‘y’ for another purpose the method ‘confint’ will return a wrong evaluation of the confidence interval without throwing any warning.
>
> I suggest to use the fixed values that are already contained within the mle object:
>
>> fit0@fullcoef[grep('y', names(fit0@fullcoef))]
> y1  y2  y3  y4  y5  y6  y7  y8  y9 y10 y11
> 26  17  13  12  20   5   9   8   5   4   8
>
> Additionally, the method ‘coef’ returns both the coefficient and the fixed parameters - while should return only the parameters - , while the ‘fullcoef’ method is not implemented - while present in the documentation of 'mle-class’.
>
> Best,
> Stefano
>
> Center for Genomic Science of IIT@SEMM
>
> Stefano de Pretis, PhD
> Postdoctoral fellow
> [hidden email] <mailto:[hidden email]>
> Via Adamello 16, 20139 Milan, Italy <http://maps.google.it/maps?q=Via+Adamello+16,+20139+Milan&oe=utf-8&client=firefox-a&ie=UTF8&hq=&hnear=Via+Adamello,+16,+20139+Milano,+Lombardia&gl=it&z=16>
> t: +39 02 94375046
>
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel