Maximum likelihood with analytical Hessian and

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Maximum likelihood with analytical Hessian and

Xavier Robin-3
Dear list,

I have an optimization problem that I would like to solve by Maximum
Likelihood.
I have analytical functions for the first and second derivatives of my
parameters.
In addition, some parameters are constrained between 0 and 1, while some
others can vary freely between -Inf and +Inf.

I am looking for an optimization function to solve this problem.

I understand that the base optim function doesn't take a Hessian
function, it only computes it numerically.
I found the maxLik package that takes the function as a "hess" parameter
but the maxNR method (the only one that uses the Hessian function) can't
be bounded.
Surprisingly I couldn't find a function doing both.

Any suggestions for a function doing bounded optimization with an
analytical Hessian function?

Thanks,
Xavier

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Maximum likelihood with analytical Hessian and

Prof J C Nash (U30A)
Of the tools I know (and things change every day!), only package trust
uses the Hessian explicitly.

It would not be too difficult to include explicit Hessian by modifying
Rvmmin which is all in R -- I'm currently doing some cleanup on that, so
ask offline if you choose that route.

Given that some parameters are between 0 and 1, you could use the
hyperbolic transformation (section 11.2 of my book Nonlinear parameter
optimization using R tools) with trust, and I think I'd try that as a
first attempt. You probably need to adjust the Hessian for the
transformation carefully.

Generally the work in computing the Hessian ( # obs * (# parameters)^2
in size) is not worth the effort, but there are problems for which it
does make a lot of sense.

JN

On 14-12-18 06:00 AM, [hidden email] wrote:

> Message: 12
> Date: Wed, 17 Dec 2014 21:46:16 +0100
> From: Xavier Robin <[hidden email]>
> To: [hidden email]
> Subject: [R] Maximum likelihood with analytical Hessian and
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset=utf-8
>
> Dear list,
>
> I have an optimization problem that I would like to solve by Maximum
> Likelihood.
> I have analytical functions for the first and second derivatives of my
> parameters.
> In addition, some parameters are constrained between 0 and 1, while some
> others can vary freely between -Inf and +Inf.
>
> I am looking for an optimization function to solve this problem.
>
> I understand that the base optim function doesn't take a Hessian
> function, it only computes it numerically.
> I found the maxLik package that takes the function as a "hess" parameter
> but the maxNR method (the only one that uses the Hessian function) can't
> be bounded.
> Surprisingly I couldn't find a function doing both.
>
> Any suggestions for a function doing bounded optimization with an
> analytical Hessian function?
>
> Thanks,
> Xavier
>
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Maximum likelihood with analytical Hessian and

Xavier Robin-3
Dear John,

Thank you for your suggestions.
I'll have a look at the trust package - the trust zone may be doing what
I need.
The tanh transformation could be a good alternative too.

Best wishes
Xavier


On 18. 12. 14 15:10, Prof J C Nash (U30A) wrote:

> Of the tools I know (and things change every day!), only package trust
> uses the Hessian explicitly.
>
> It would not be too difficult to include explicit Hessian by modifying
> Rvmmin which is all in R -- I'm currently doing some cleanup on that, so
> ask offline if you choose that route.
>
> Given that some parameters are between 0 and 1, you could use the
> hyperbolic transformation (section 11.2 of my book Nonlinear parameter
> optimization using R tools) with trust, and I think I'd try that as a
> first attempt. You probably need to adjust the Hessian for the
> transformation carefully.
>
> Generally the work in computing the Hessian ( # obs * (# parameters)^2
> in size) is not worth the effort, but there are problems for which it
> does make a lot of sense.
>
> JN
>
> On 14-12-18 06:00 AM, [hidden email] wrote:
>> Message: 12
>> Date: Wed, 17 Dec 2014 21:46:16 +0100
>> From: Xavier Robin <[hidden email]>
>> To: [hidden email]
>> Subject: [R] Maximum likelihood with analytical Hessian and
>> Message-ID: <[hidden email]>
>> Content-Type: text/plain; charset=utf-8
>>
>> Dear list,
>>
>> I have an optimization problem that I would like to solve by Maximum
>> Likelihood.
>> I have analytical functions for the first and second derivatives of my
>> parameters.
>> In addition, some parameters are constrained between 0 and 1, while some
>> others can vary freely between -Inf and +Inf.
>>
>> I am looking for an optimization function to solve this problem.
>>
>> I understand that the base optim function doesn't take a Hessian
>> function, it only computes it numerically.
>> I found the maxLik package that takes the function as a "hess" parameter
>> but the maxNR method (the only one that uses the Hessian function) can't
>> be bounded.
>> Surprisingly I couldn't find a function doing both.
>>
>> Any suggestions for a function doing bounded optimization with an
>> analytical Hessian function?
>>
>> Thanks,
>> Xavier
>>
>>


--
Xavier Robin, PhD
Cellular Signal Integration Group (C-SIG)  - Linding Lab
Biotech Research and Innovation Center (BRIC) - University of Copenhagen
Anker Engelundsvej, DTU Campus, Building 301, DK-2800 Lyngby, DENMARK
Mobile: +45 42 799 833
www.lindinglab.org - www.bric.ku.dk

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.