Non-negativity constraints for logistic regression

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Non-negativity constraints for logistic regression

NightlordTW
Dear R users,

I am currently attempting to fit logistic regression models in R, where
the slopes should be restricted to positive values. Although I am aware
of the package nnls (which does the trick for linear regression models),
I did not find any solution for logistic regression. If there is any
package available for this purpose, I would be interested to know them.

Alternatively, I realize it is possible to optimize a specialized
likelihood function that does the trick. Although I know how to optimize
the log-likelihood of logistic regression models, I am not sure how to
implement non-negativity constraints for slope parameters without
messing up the Newton optimization. Therefore, I am also interested in
solutions for this problem.

Best regards,

Thomas Debray

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-negativity constraints for logistic regression

Prof Brian Ripley
On 21/12/2011 18:26, [hidden email] wrote:
> Dear R users,
>
> I am currently attempting to fit logistic regression models in R, where
> the slopes should be restricted to positive values. Although I am aware

I guess non-negative, as in the subject line, so there actually is a
solution.

> of the package nnls (which does the trick for linear regression models),
> I did not find any solution for logistic regression. If there is any
> package available for this purpose, I would be interested to know them.
>
> Alternatively, I realize it is possible to optimize a specialized
> likelihood function that does the trick. Although I know how to optimize
> the log-likelihood of logistic regression models, I am not sure how to
> implement non-negativity constraints for slope parameters without
> messing up the Newton optimization. Therefore, I am also interested in
> solutions for this problem.

There is an example of this in the 'Optimization' chapter of MASS (the
book, page 445 to be precise).  You simply use an optimizer with box
constraints: see ?optim and ?nlminb, for example.

> Best regards,
>
> Thomas Debray


--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-negativity constraints for logistic regression

NightlordTW
In reply to this post by NightlordTW
> Dear R users,
> I am currently attempting to fit logistic regression models in R, where
> the slopes should be restricted to positive values. Although I am aware
>> I guess non-negative, as in the subject line, so there actually is a
solution.

Indeed, I meant non-negative, zero slopes are also possible parameter
values for my case.


> of the package nnls (which does the trick for linear regression models),
> I did not find any solution for logistic regression. If there is any
> package available for this purpose, I would be interested to know them.
> Alternatively, I realize it is possible to optimize a specialized
> likelihood function that does the trick. Although I know how to optimize
> the log-likelihood of logistic regression models, I am not sure how to
> implement non-negativity constraints for slope parameters without
> messing up the Newton optimization. Therefore, I am also interested in
> solutions for this problem.

>> There is an example of this in the 'Optimization' chapter of MASS (the
book, page 445 to be precise).  You simply use an optimizer with box
constraints: see ?optim and ?nlminb, for example.

Thanks a lot, I managed to get it fully working by passing the constraints
to L-BFGS-B.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-negativity constraints for logistic regression

Ravi Varadhan
In reply to this post by NightlordTW
Hi Thomas,

Using box-constrained optimizer in glm.fit is a good suggestion for finding the point estimates.  However, there is still the issue of making inference, i.e., computing the variances and p-values for the estimates.  You have to deal with the issue of MLE possibly being on the boundary.  Asymptotic distribution of MLE estimators will not be normal in the case of convergence at the boundary.  This is a difficult problem.

Best,
Ravi

-------------------------------------------------------
Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University

Ph. (410) 502-2619
email: [hidden email]<mailto:[hidden email]>


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.