Suggestion for the optimization code

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Suggestion for the optimization code

Mathieu Ribatet
    Dear list,

Here's a suggestion about the different optimization code. There are
several optimization procedures in the base package (optim, optimize,
nlm, nlminb, ..). However, the output of these functions are slightly
different. For instance,

   1. optim returns a list with arguments par (the estimates), value the
      minimum (maxima) of the objective function, convergence (optim
      .convergence)
   2. optimize returns a list with arguments minimum (or maximum) giving
      the estimates, objective the value of the obj. function
   3. nlm returns a list with arguments minimum giving the minimum of
      the obj. function, minimum the estimates, code the optim. convergence
   4. nlminb returns a list with arguments par (the estimates),
      objective, convergence (conv. code), evaluations

Furthermore, optim keeps the names of the parameters while nlm, nlminb
don't.
s
I believe it would be nice if all these optimizers have a kind of
homogenized output. This will help in writing functions that can call
different optimizers. Obviously, we can write our own function that
homogenized the output after calling the optimizer, but I still believe
this will be more user-friendly.

Do you think this is a reasonable feature to implement - despite it
isn't an important point?
Best,
Mathieu

* BTW, if this is relevant, I could try to do it.

--
Institute of Mathematics
Ecole Polytechnique Fédérale de Lausanne
STAT-IMA-FSB-EPFL, Station 8
CH-1015 Lausanne   Switzerland
http://stat.epfl.ch/
Tel: + 41 (0)21 693 7907

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion for the optimization code

Duncan Murdoch
On 8/8/2008 8:56 AM, Mathieu Ribatet wrote:

>     Dear list,
>
> Here's a suggestion about the different optimization code. There are
> several optimization procedures in the base package (optim, optimize,
> nlm, nlminb, ..). However, the output of these functions are slightly
> different. For instance,
>
>    1. optim returns a list with arguments par (the estimates), value the
>       minimum (maxima) of the objective function, convergence (optim
>       .convergence)
>    2. optimize returns a list with arguments minimum (or maximum) giving
>       the estimates, objective the value of the obj. function
>    3. nlm returns a list with arguments minimum giving the minimum of
>       the obj. function, minimum the estimates, code the optim. convergence
>    4. nlminb returns a list with arguments par (the estimates),
>       objective, convergence (conv. code), evaluations
>
> Furthermore, optim keeps the names of the parameters while nlm, nlminb
> don't.
> s
> I believe it would be nice if all these optimizers have a kind of
> homogenized output. This will help in writing functions that can call
> different optimizers. Obviously, we can write our own function that
> homogenized the output after calling the optimizer, but I still believe
> this will be more user-friendly.

Unfortunately, changing the names within the return value would break a
lot of existing uses of those functions.  Writing a wrapper to
homogenize the output is probably the right thing to do.

Duncan Murdoch

> Do you think this is a reasonable feature to implement - despite it
> isn't an important point?
> Best,
> Mathieu
>
> * BTW, if this is relevant, I could try to do it.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion for the optimization code

Prof Brian Ripley
In reply to this post by Mathieu Ribatet
On Fri, 8 Aug 2008, Mathieu Ribatet wrote:

>   Dear list,
>
> Here's a suggestion about the different optimization code. There are several
> optimization procedures in the base package (optim, optimize, nlm, nlminb,
> ..). However, the output of these functions are slightly different. For
> instance,
>
>  1. optim returns a list with arguments par (the estimates), value the
>     minimum (maxima) of the objective function, convergence (optim
>     .convergence)
>  2. optimize returns a list with arguments minimum (or maximum) giving
>     the estimates, objective the value of the obj. function
>  3. nlm returns a list with arguments minimum giving the minimum of
>     the obj. function, minimum the estimates, code the optim. convergence
>  4. nlminb returns a list with arguments par (the estimates),
>     objective, convergence (conv. code), evaluations
>
> Furthermore, optim keeps the names of the parameters while nlm, nlminb don't.
> s
> I believe it would be nice if all these optimizers have a kind of homogenized
> output. This will help in writing functions that can call different
> optimizers. Obviously, we can write our own function that homogenized the
> output after calling the optimizer, but I still believe this will be more
> user-friendly.
>
> Do you think this is a reasonable feature to implement - despite it isn't an
> important point?
This would be essentially impossible without breaking most existing code,
and in the case of optimize() and nlminb() that goes back many years to
uses in S(-PLUS).

> Best,
> Mathieu
>
> * BTW, if this is relevant, I could try to do it.
>
> --
> Institute of Mathematics
> Ecole Polytechnique Fédérale de Lausanne
> STAT-IMA-FSB-EPFL, Station 8
> CH-1015 Lausanne   Switzerland
> http://stat.epfl.ch/
> Tel: + 41 (0)21 693 7907
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion for the optimization code

Robert Gentleman
In reply to this post by Duncan Murdoch


Duncan Murdoch wrote:

> On 8/8/2008 8:56 AM, Mathieu Ribatet wrote:
>>     Dear list,
>>
>> Here's a suggestion about the different optimization code. There are
>> several optimization procedures in the base package (optim, optimize,
>> nlm, nlminb, ..). However, the output of these functions are slightly
>> different. For instance,
>>
>>    1. optim returns a list with arguments par (the estimates), value the
>>       minimum (maxima) of the objective function, convergence (optim
>>       .convergence)
>>    2. optimize returns a list with arguments minimum (or maximum) giving
>>       the estimates, objective the value of the obj. function
>>    3. nlm returns a list with arguments minimum giving the minimum of
>>       the obj. function, minimum the estimates, code the optim.
>> convergence
>>    4. nlminb returns a list with arguments par (the estimates),
>>       objective, convergence (conv. code), evaluations
>>
>> Furthermore, optim keeps the names of the parameters while nlm, nlminb
>> don't.
>> s
>> I believe it would be nice if all these optimizers have a kind of
>> homogenized output. This will help in writing functions that can call
>> different optimizers. Obviously, we can write our own function that
>> homogenized the output after calling the optimizer, but I still
>> believe this will be more user-friendly.
>
> Unfortunately, changing the names within the return value would break a
> lot of existing uses of those functions.  Writing a wrapper to
> homogenize the output is probably the right thing to do.

   And potentially to harmonize inputs. The MLInterfaces package
(Bioconductor) has done this for many machine learning algorithms,
should you want an example to look at.

   Robert


>
> Duncan Murdoch
>
>> Do you think this is a reasonable feature to implement - despite it
>> isn't an important point?
>> Best,
>> Mathieu
>>
>> * BTW, if this is relevant, I could try to do it.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

--
Robert Gentleman, PhD
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M2-B876
PO Box 19024
Seattle, Washington 98109-1024
206-667-7700
[hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion for the optimization code

Mathieu Ribatet
In reply to this post by Prof Brian Ripley
Ok, please consider it as a bad call.
Thanks for your answers.
Best,
Mathieu

Prof Brian Ripley a écrit :

> On Fri, 8 Aug 2008, Mathieu Ribatet wrote:
>
>  
>>   Dear list,
>>
>> Here's a suggestion about the different optimization code. There are several
>> optimization procedures in the base package (optim, optimize, nlm, nlminb,
>> ..). However, the output of these functions are slightly different. For
>> instance,
>>
>>  1. optim returns a list with arguments par (the estimates), value the
>>     minimum (maxima) of the objective function, convergence (optim
>>     .convergence)
>>  2. optimize returns a list with arguments minimum (or maximum) giving
>>     the estimates, objective the value of the obj. function
>>  3. nlm returns a list with arguments minimum giving the minimum of
>>     the obj. function, minimum the estimates, code the optim. convergence
>>  4. nlminb returns a list with arguments par (the estimates),
>>     objective, convergence (conv. code), evaluations
>>
>> Furthermore, optim keeps the names of the parameters while nlm, nlminb don't.
>> s
>> I believe it would be nice if all these optimizers have a kind of homogenized
>> output. This will help in writing functions that can call different
>> optimizers. Obviously, we can write our own function that homogenized the
>> output after calling the optimizer, but I still believe this will be more
>> user-friendly.
>>
>> Do you think this is a reasonable feature to implement - despite it isn't an
>> important point?
>>    
>
> This would be essentially impossible without breaking most existing code,
> and in the case of optimize() and nlminb() that goes back many years to
> uses in S(-PLUS).
>
>  
>> Best,
>> Mathieu
>>
>> * BTW, if this is relevant, I could try to do it.
>>
>> --
>> Institute of Mathematics
>> Ecole Polytechnique Fédérale de Lausanne
>> STAT-IMA-FSB-EPFL, Station 8
>> CH-1015 Lausanne   Switzerland
>> http://stat.epfl.ch/
>> Tel: + 41 (0)21 693 7907
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>    
>
> --
> Brian D. Ripley,                  [hidden email]
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>  

--
Institute of Mathematics
Ecole Polytechnique Fédérale de Lausanne
STAT-IMA-FSB-EPFL, Station 8
CH-1015 Lausanne   Switzerland
http://stat.epfl.ch/
Tel: + 41 (0)21 693 7907

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion for the optimization code

Ben Bolker


Mathieu Ribatet <mathieu.ribatet <at> epfl.ch> writes:

>
> Ok, please consider it as a bad call.
> Thanks for your answers.
> Best,
> Mathieu
>

  Well, I don't think it's a _bad_ call; I think the
underlying wish (more flexibility in moving between
existing optimizers without changing the objective
function, calls, etc.) is valid -- but can really
only be achieved at this point by writing a wrapper
(Optimize(), or is that too confusing?), because
of backward compatibility issues.  I would also
like to see a more open framework in optim() [or
elsewhere], where one can more easily plug in
alternative optimization procedures.

  My version of mle (mle2, in bbmle) does something
like this, but in an ad hoc way -- it can use optim,
nlm, nlminb, or constrOptim as an optimization backend.
(I will also take a look at Robert Gentleman's code,
now ...)

  Ben Bolker

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel