Non-finite finite difference error

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Non-finite finite difference error

JS.Augustyn
Hello, I'm trying to use fitdistr() from the MASS package to fit a gamma  
distribution to a set of data. The data set is too large (1167 values) to  
reproduce in an email, but the summary statistics are:

Min. 1st Qu. Median Mean 3rd Qu. Max.
116.7 266.7 666.7 1348.0 1642.0 16720.0

The call I'm trying to make is:
fitdistr(x,"gamma")

and the error is:
Error in optim(x = c(3466.676842, 1666.749002, 2500.067852, 1200.053892, :
non-finite finite-difference value [2]
In addition: Warning message:
In dgamma(x, shape, scale, log) : NaNs produced

I found a couple of other posts from folks who were getting the same error  
from optim(), but did not find any useful tips for my situation. The error  
seems to indicate a problem with value 2 in my data set (1666.749002), but  
nothing seems odd about that value.

I'm willing to pass along the full data set as an attachment if it would  
help.

Thank you in advance!

Jason S. Augustyn, Ph.D.

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-finite finite difference error

Ravi Varadhan
Hi Jason,

The error message indicates that there was problem in estimating the gradient of objective function.  It has nothing to do with your second data point.  This could happen for a variety of reasons, but the most proximate cause of the problem seems due to a parameter being negative during the iteration.  The best solution, most often, is to provide a better (or atleast a different) starting value.  Look at the example in the help page for "fitdistr":

fitdistr(x, dgamma, list(shape = 1, rate = 0.1), lower = 0.01)

Note that the above command specifies lower bounds on both the shape and the rate parameter (hence a different optimziation algorithm will be used in "optim").

Best,
Ravi.
____________________________________________________________________

Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology
School of Medicine
Johns Hopkins University

Ph. (410) 502-2619
email: [hidden email]


----- Original Message -----
From: [hidden email]
Date: Tuesday, December 23, 2008 11:27 pm
Subject: [R] Non-finite finite difference error
To: [hidden email]


> Hello, I'm trying to use fitdistr() from the MASS package to fit a
> gamma  
>  distribution to a set of data. The data set is too large (1167
> values) to  
>  reproduce in an email, but the summary statistics are:
>  
>  Min. 1st Qu. Median Mean 3rd Qu. Max.
>  116.7 266.7 666.7 1348.0 1642.0 16720.0
>  
>  The call I'm trying to make is:
>  fitdistr(x,"gamma")
>  
>  and the error is:
>  Error in optim(x = c(3466.676842, 1666.749002, 2500.067852,
> 1200.053892, :
>  non-finite finite-difference value [2]
>  In addition: Warning message:
>  In dgamma(x, shape, scale, log) : NaNs produced
>  
>  I found a couple of other posts from folks who were getting the same
> error  
>  from optim(), but did not find any useful tips for my situation. The
> error  
>  seems to indicate a problem with value 2 in my data set
> (1666.749002), but  
>  nothing seems odd about that value.
>  
>  I'm willing to pass along the full data set as an attachment if it
> would  
>  help.
>  
>  Thank you in advance!
>  
>  Jason S. Augustyn, Ph.D.
>  
>   [[alternative HTML version deleted]]
>  
>  ______________________________________________
>  [hidden email] mailing list
>  
>  PLEASE do read the posting guide
>  and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-finite finite difference error

JS.Augustyn
Ravi, thank you for the explanation - that makes perfect sense, and it
wouldn't have occurred to me to suspect a negative parameter based on the
error. Using the elaborated syntax from the help page yields a lovely fit.
Regards,

Jason



On Wed, Dec 24, 2008 at 10:57 AM, Ravi Varadhan <[hidden email]> wrote:

> Hi Jason,
>
> The error message indicates that there was problem in estimating the
> gradient of objective function.  It has nothing to do with your second data
> point.  This could happen for a variety of reasons, but the most proximate
> cause of the problem seems due to a parameter being negative during the
> iteration.  The best solution, most often, is to provide a better (or
> atleast a different) starting value.  Look at the example in the help page
> for "fitdistr":
>
> fitdistr(x, dgamma, list(shape = 1, rate = 0.1), lower = 0.01)
>
> Note that the above command specifies lower bounds on both the shape and
> the rate parameter (hence a different optimziation algorithm will be used in
> "optim").
>
> Best,
> Ravi.
> ____________________________________________________________________
>
> Ravi Varadhan, Ph.D.
> Assistant Professor,
> Division of Geriatric Medicine and Gerontology
> School of Medicine
> Johns Hopkins University
>
> Ph. (410) 502-2619
> email: [hidden email]
>
>
> ----- Original Message -----
> From: [hidden email]
> Date: Tuesday, December 23, 2008 11:27 pm
> Subject: [R] Non-finite finite difference error
> To: [hidden email]
>
>
> > Hello, I'm trying to use fitdistr() from the MASS package to fit a
> > gamma
> >  distribution to a set of data. The data set is too large (1167
> > values) to
> >  reproduce in an email, but the summary statistics are:
> >
> >  Min. 1st Qu. Median Mean 3rd Qu. Max.
> >  116.7 266.7 666.7 1348.0 1642.0 16720.0
> >
> >  The call I'm trying to make is:
> >  fitdistr(x,"gamma")
> >
> >  and the error is:
> >  Error in optim(x = c(3466.676842, 1666.749002, 2500.067852,
> > 1200.053892, :
> >  non-finite finite-difference value [2]
> >  In addition: Warning message:
> >  In dgamma(x, shape, scale, log) : NaNs produced
> >
> >  I found a couple of other posts from folks who were getting the same
> > error
> >  from optim(), but did not find any useful tips for my situation. The
> > error
> >  seems to indicate a problem with value 2 in my data set
> > (1666.749002), but
> >  nothing seems odd about that value.
> >
> >  I'm willing to pass along the full data set as an attachment if it
> > would
> >  help.
> >
> >  Thank you in advance!
> >
> >  Jason S. Augustyn, Ph.D.
> >
> >       [[alternative HTML version deleted]]
> >
> >  ______________________________________________
> >  [hidden email] mailing list
> >
> >  PLEASE do read the posting guide
> >  and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-finite finite difference error

hartawan
This post has NOT been accepted by the mailing list yet.
In reply to this post by JS.Augustyn
 Hello, I'm trying to use MLE from the MASS and  stats4 package to fit a gompertz  
distribution to a set of data. The data set is too large (11626 values) to  
reproduce in an email, this is my script program:
"
program=function(data,B1,B2,C1,C2,B3,B4,C3,C4,mu){
x=data$M
y=data$F
v=data$v
d1=data$d1
d2=data$d2
d3=data$d3
u1=data$u1
u2=data$u2
h1=data$h1
h2=data$h2

library(MASS)
library(stats4)

est=function(B1,B2,C1,C2,B3,B4,C3,C4,mu){
n=nrow(data)
-sum((-(B1*C1^x*(C1^v-1))/log(C1))+
(d2*log(B1*C1^(x+v)))-
((B2*C2^y*(C2^v-1))/log(C2))+
(d1*log(B2*C2^(y+v)))-
v*mu+
d3*log(mu)-((B3*(C3^(x+v))*(C3^u1-1))/log(C3))+h1*log(B3*C3^(x+v+u1))-((B4*(C4^(y+v))*(C4^u2-1))/log(C4))+h2*log(B4*C4^(y+v+u2)))
}
estmle=mle(minuslogl=est,
start=list(B1=B1,B2=B2,C1=C1,C2=C2,B3=B3,B4=B4,C3=C3,C4=C4,mu=mu))
summary(estmle)
}

program(data,B1=0.1,B2=0.1,C1=1.1,C2=1.1,B3=0.1,B4=0.1,C3=1.1,C4=1.1,mu=0.001407)

the result are:

Error in optim(start, f, method = method, hessian = TRUE, ...) :
  non-finite finite-difference value [5]
 
In addition: Warning message:
Warning in log(C1) : NaNs produced
Warning in log(B1 * C1^(x + v)) : NaNs produced
Warning in log(C2) : NaNs produced
Warning in log(B2 * C2^(y + v)) : NaNs produced
Warning in log(C3) : NaNs produced
(there are 50 warnings)

I'm willing to pass along the full data set as an attachment if it would  
help.

Thank you in advance!

yudi hartawan