optim with upper and lower bounds

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

optim with upper and lower bounds

R help mailing list-2
I am running a lot of optimization problems, at the moment using 'optim'  ('optim' is actually called by another program).  All of the problems have variables with simple upper and lower bounds,  which I can easily transform into a form that is unconstrained and solve using 'BFGS'.  But I was wondering is if it is more robust to solve the problem this way,  or to use L-BFGS-B instead.

Also how much difference can it make using 'optimx' instead 'optim'?  The program I am using (KFAS) allows this,  I just have to do some extra programming to use it.

Thanks,

-Roy



**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: [hidden email] www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: optim with upper and lower bounds

Peter Dalgaard-2
This stuff is of course dependent on exactly which optimization problem you have, but optimx::optimr is often a very good drop-in replacement for optim, especially when bounds are involved (e.g., optim has an awkward habit of attempting evaluations outside the domain when numerical derivatives are needed).

You might want to look at the last examples in ?stats4::mle (in R 4.x.x)

-pd

> On 10 Aug 2020, at 22:08 , Roy Mendelssohn - NOAA Federal via R-help <[hidden email]> wrote:
>
> I am running a lot of optimization problems, at the moment using 'optim'  ('optim' is actually called by another program).  All of the problems have variables with simple upper and lower bounds,  which I can easily transform into a form that is unconstrained and solve using 'BFGS'.  But I was wondering is if it is more robust to solve the problem this way,  or to use L-BFGS-B instead.
>
> Also how much difference can it make using 'optimx' instead 'optim'?  The program I am using (KFAS) allows this,  I just have to do some extra programming to use it.
>
> Thanks,
>
> -Roy
>
>
>
> **********************
> "The contents of this message do not reflect any position of the U.S. Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: [hidden email] www: https://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: optim with upper and lower bounds

J C Nash
Thanks to Peter for noting that the numerical derivative part of code doesn't check bounds in optim().
I tried to put some checks into Rvmmin and Rcgmin in optimx package (they were separate packages before, and
still on CRAN), but I'm far from capturing all the places where numerical derivative steps can go outside bounds.

And if you have a "production" problem where you are going to run a given optimization over a lot of cases, I'd
strongly suggest that you write your own derivative code, even if it is a numerical approximation. In the case of
a specialized derivative code e.g., part analytic, part numeric, with bounds checking, I'll be willing
to kibbitz, but suggest off-list until something is working, in which case it is probably at least worth
a vignette, as this sort of situation seems to pop up at least once a year and a good example would really
be helpful to guide the process. I'm reluctant to prepare an artificial example because, well, it will be
artificial and not capture the sort of details that have to be addressed.

Best, JN


On 2020-08-11 3:48 a.m., peter dalgaard wrote:

> This stuff is of course dependent on exactly which optimization problem you have, but optimx::optimr is often a very good drop-in replacement for optim, especially when bounds are involved (e.g., optim has an awkward habit of attempting evaluations outside the domain when numerical derivatives are needed).
>
> You might want to look at the last examples in ?stats4::mle (in R 4.x.x)
>
> -pd
>
>> On 10 Aug 2020, at 22:08 , Roy Mendelssohn - NOAA Federal via R-help <[hidden email]> wrote:
>>
>> I am running a lot of optimization problems, at the moment using 'optim'  ('optim' is actually called by another program).  All of the problems have variables with simple upper and lower bounds,  which I can easily transform into a form that is unconstrained and solve using 'BFGS'.  But I was wondering is if it is more robust to solve the problem this way,  or to use L-BFGS-B instead.
>>
>> Also how much difference can it make using 'optimx' instead 'optim'?  The program I am using (KFAS) allows this,  I just have to do some extra programming to use it.
>>
>> Thanks,
>>
>> -Roy
>>
>>
>>
>> **********************
>> "The contents of this message do not reflect any position of the U.S. Government or NOAA."
>> **********************
>> Roy Mendelssohn
>> Supervisory Operations Research Analyst
>> NOAA/NMFS
>> Environmental Research Division
>> Southwest Fisheries Science Center
>> ***Note new street address***
>> 110 McAllister Way
>> Santa Cruz, CA 95060
>> Phone: (831)-420-3666
>> Fax: (831) 420-3980
>> e-mail: [hidden email] www: https://www.pfeg.noaa.gov/
>>
>> "Old age and treachery will overcome youth and skill."
>> "From those who have been given much, much will be expected"
>> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: optim with upper and lower bounds

R help mailing list-2
Thanks to all who responded.  Will take me some time to digest it all.

-Roy


> On Aug 11, 2020, at 6:24 AM, J C Nash <[hidden email]> wrote:
>
> Thanks to Peter for noting that the numerical derivative part of code doesn't check bounds in optim().
> I tried to put some checks into Rvmmin and Rcgmin in optimx package (they were separate packages before, and
> still on CRAN), but I'm far from capturing all the places where numerical derivative steps can go outside bounds.
>
> And if you have a "production" problem where you are going to run a given optimization over a lot of cases, I'd
> strongly suggest that you write your own derivative code, even if it is a numerical approximation. In the case of
> a specialized derivative code e.g., part analytic, part numeric, with bounds checking, I'll be willing
> to kibbitz, but suggest off-list until something is working, in which case it is probably at least worth
> a vignette, as this sort of situation seems to pop up at least once a year and a good example would really
> be helpful to guide the process. I'm reluctant to prepare an artificial example because, well, it will be
> artificial and not capture the sort of details that have to be addressed.
>
> Best, JN
>
>
> On 2020-08-11 3:48 a.m., peter dalgaard wrote:
>> This stuff is of course dependent on exactly which optimization problem you have, but optimx::optimr is often a very good drop-in replacement for optim, especially when bounds are involved (e.g., optim has an awkward habit of attempting evaluations outside the domain when numerical derivatives are needed).
>>
>> You might want to look at the last examples in ?stats4::mle (in R 4.x.x)
>>
>> -pd
>>
>>> On 10 Aug 2020, at 22:08 , Roy Mendelssohn - NOAA Federal via R-help <[hidden email]> wrote:
>>>
>>> I am running a lot of optimization problems, at the moment using 'optim'  ('optim' is actually called by another program).  All of the problems have variables with simple upper and lower bounds,  which I can easily transform into a form that is unconstrained and solve using 'BFGS'.  But I was wondering is if it is more robust to solve the problem this way,  or to use L-BFGS-B instead.
>>>
>>> Also how much difference can it make using 'optimx' instead 'optim'?  The program I am using (KFAS) allows this,  I just have to do some extra programming to use it.
>>>
>>> Thanks,
>>>
>>> -Roy
>>>
>>>
>>>
>>> **********************
>>> "The contents of this message do not reflect any position of the U.S. Government or NOAA."
>>> **********************
>>> Roy Mendelssohn
>>> Supervisory Operations Research Analyst
>>> NOAA/NMFS
>>> Environmental Research Division
>>> Southwest Fisheries Science Center
>>> ***Note new street address***
>>> 110 McAllister Way
>>> Santa Cruz, CA 95060
>>> Phone: (831)-420-3666
>>> Fax: (831) 420-3980
>>> e-mail: [hidden email] www: https://www.pfeg.noaa.gov/
>>>
>>> "Old age and treachery will overcome youth and skill."
>>> "From those who have been given much, much will be expected"
>>> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>>>
>>> ______________________________________________
>>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: [hidden email] www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.