survreg's algorithm

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

survreg's algorithm

Gad Abraham-3
Hi,

I'm using survreg() from the survival package for parametric survival
regression (modelling inter-arrival times of patients to a waiting list
as exponentially distributed, with various regressors such as queue size
and season).

Does anyone know which algorithm survreg() uses for this?

Thanks,
Gad

--
Gad Abraham
Department of Mathematics and Statistics
The University of Melbourne
Parkville 3010, Victoria, Australia
email: [hidden email]
web: http://www.ms.unimelb.edu.au/~gabraham

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: survreg's algorithm

Gad Abraham-3
Gad Abraham wrote:

> Hi,
>
> I'm using survreg() from the survival package for parametric survival
> regression (modelling inter-arrival times of patients to a waiting list
> as exponentially distributed, with various regressors such as queue size
> and season).
>
> Does anyone know which algorithm survreg() uses for this?
>
> Thanks,
> Gad
>

Due diligence:

I have actually looked at ?survreg and friends, and at the source code;
except for a brief mention of a "sparse Newton-Rapshon algorithm" in the
frailty model code, I couldn't find anything substantive.


--
Gad Abraham
Department of Mathematics and Statistics
The University of Melbourne
Parkville 3010, Victoria, Australia
email: [hidden email]
web: http://www.ms.unimelb.edu.au/~gabraham

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: survreg's algorithm

Simon Blomberg-4
Did you look at the C source code? There are 4 different variants
(survregN.c, where N <- 2:5) , depending on whether the distribution is
built-in or not, and penalized likelihood is being used or not. They all
look like NR to me, but I confess I haven't read the code in extreme
detail. It is well commented, however.

Cheers,

Simon.

PS Try figuring out what <insert commercial statistical software here>
is actually doing by reading the source code. Take that, large software
corporations! The future belongs to R!

On Tue, 2007-10-16 at 13:12 +1000, Gad Abraham wrote:

> Gad Abraham wrote:
> > Hi,
> >
> > I'm using survreg() from the survival package for parametric survival
> > regression (modelling inter-arrival times of patients to a waiting list
> > as exponentially distributed, with various regressors such as queue size
> > and season).
> >
> > Does anyone know which algorithm survreg() uses for this?
> >
> > Thanks,
> > Gad
> >
>
> Due diligence:
>
> I have actually looked at ?survreg and friends, and at the source code;
> except for a brief mention of a "sparse Newton-Rapshon algorithm" in the
> frailty model code, I couldn't find anything substantive.
>
>
--
Simon Blomberg, BSc (Hons), PhD, MAppStat.
Lecturer and Consultant Statistician
Faculty of Biological and Chemical Sciences
The University of Queensland
St. Lucia Queensland 4072
Australia
Room 320 Goddard Building (8)
T: +61 7 3365 2506
email: S.Blomberg1_at_uq.edu.au

Policies:
1.  I will NOT analyse your data for you.
2.  Your deadline is your problem.

The combination of some data and an aching desire for
an answer does not ensure that a reasonable answer can
be extracted from a given body of data. - John Tukey.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: survreg's algorithm

Therneau, Terry M., Ph.D.
In reply to this post by Gad Abraham-3
Gad Abraham wrote:

> Hi,
>
> I'm using survreg() from the survival package for parametric survival
> regression (modelling inter-arrival times of patients to a waiting list
> as exponentially distributed, with various regressors such as queue size
> and season).
>
> Does anyone know which algorithm survreg() uses for this?
>
> Thanks,
> Gad

  The code uses Newton-Raphson iteration, dropping back to a Marquardt step if
it has trouble with convergence.  (One data set in the test suite, sent in by a
user, has a likelihood whose contours look like a lima bean.  This was fatal to
the simple NR).
 
  There are several technical reports at the Mayo bistatistics web site that
document various portions of the survival code in depth. I need to add an index
to them to the documentation posted in the R package

  http://mayoresearch.mayo.edu/mayo/research/biostat/
 
In particular, #53 contains information on the parametric models that is not
found elsewhere.  

        Terry Therneau

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: survreg's algorithm

Thomas Lumley
In reply to this post by Gad Abraham-3
On Tue, 16 Oct 2007, Gad Abraham wrote:

> Gad Abraham wrote:
>> Hi,
>>
>> I'm using survreg() from the survival package for parametric survival
>> regression (modelling inter-arrival times of patients to a waiting list
>> as exponentially distributed, with various regressors such as queue size
>> and season).
>>
>> Does anyone know which algorithm survreg() uses for this?
>>

It uses the Newton-Raphson algorithm. Since the loglikelihood has a unique
maximum it shouldn't really matter which algorithm is being used (for this
model -- some of the other models are a bit touchy with small data sets),
so perhaps you are asking what estimator survreg() uses?  It uses
the maximum likelihood estimator.

  -thomas

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: survreg's algorithm

Gad Abraham-3
In reply to this post by Therneau, Terry M., Ph.D.
Terry Therneau wrote:

> Gad Abraham wrote:
>> Hi,
>>
>> I'm using survreg() from the survival package for parametric survival
>> regression (modelling inter-arrival times of patients to a waiting list
>> as exponentially distributed, with various regressors such as queue size
>> and season).
>>
>> Does anyone know which algorithm survreg() uses for this?
>>
>> Thanks,
>> Gad
>
>   The code uses Newton-Raphson iteration, dropping back to a Marquardt step if
> it has trouble with convergence.  (One data set in the test suite, sent in by a
> user, has a likelihood whose contours look like a lima bean.  This was fatal to
> the simple NR).
>  
>   There are several technical reports at the Mayo bistatistics web site that
> document various portions of the survival code in depth. I need to add an index
> to them to the documentation posted in the R package
>
>   http://mayoresearch.mayo.edu/mayo/research/biostat/
>  
> In particular, #53 contains information on the parametric models that is not
> found elsewhere.  
>
> Terry Therneau
>

Thanks Terry,

The description in Section 5.7 "Fitting the model" is just what I was
looking for.

Cheers,
Gad

--
Gad Abraham
Department of Mathematics and Statistics
The University of Melbourne
Parkville 3010, Victoria, Australia
email: [hidden email]
web: http://www.ms.unimelb.edu.au/~gabraham

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: survreg's algorithm

Gad Abraham-3
In reply to this post by Thomas Lumley
Thomas Lumley wrote:

> On Tue, 16 Oct 2007, Gad Abraham wrote:
>
>> Gad Abraham wrote:
>>> Hi,
>>>
>>> I'm using survreg() from the survival package for parametric survival
>>> regression (modelling inter-arrival times of patients to a waiting list
>>> as exponentially distributed, with various regressors such as queue size
>>> and season).
>>>
>>> Does anyone know which algorithm survreg() uses for this?
>>>
>
> It uses the Newton-Raphson algorithm. Since the loglikelihood has a
> unique maximum it shouldn't really matter which algorithm is being used
> (for this model -- some of the other models are a bit touchy with small
> data sets), so perhaps you are asking what estimator survreg() uses?  It
> uses the maximum likelihood estimator.
>
>     -thomas


Thanks Simon and Thomas.

Terry Therneau has indeed clarified that it is maximising the
loglikelihood using Newton-Raphson, as explained in Section 5.7 of
http://mayoresearch.mayo.edu/mayo/research/biostat/upload/survival.pdf

Cheers,
Gad

--
Gad Abraham
Department of Mathematics and Statistics
The University of Melbourne
Parkville 3010, Victoria, Australia
email: [hidden email]
web: http://www.ms.unimelb.edu.au/~gabraham

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.