# Failure of solve.QP in portfolio modeling

6 messages
Open this post in threaded view
|

## Failure of solve.QP in portfolio modeling

 Hi everyone, I'm trying to estimate optimal Markowitz portfolio weights for a list of stocks chosen upon some criterion using solve.QP from quadprog library. When the number of stocks N reaches some limit, I get a message "matrix D in quadratic function is not positive definite." For example, if I rebalance every 6 weeks (which implies that variance is calculated for 6-week interval prior to the period for which I calculate portfolio weights), I can get solution for 25>=N<50. For 12-week interval, solution exists for 50>=N<100, and for 24-week interval, I can get solution for N=100. My attempt to remedy this problem with Higham's method doesn't help. I'll greatly appreciate you input: first, why this may happen (can there be lack of local minimum?), and second, whether there are R solvers that may need only semi positive definite matrix. Thanks! Alec _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance-- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
Open this post in threaded view
|

## Re: Failure of solve.QP in portfolio modeling

 Because you can't invert a matrix with more features than observations. IE if you have 50 stocks but only use 26 weekly return data points, you can't invert that covariance matrix because it's not positive semi-definite. You may want to try my Constrained Critical Line algorithm. https://quantstrattrader.wordpress.com/2015/06/05/momentum-markowitz-and-solving-rank-deficient-covariance-matrices-the-constrained-critical-line-algorithm/On Tue, Sep 22, 2015 at 9:37 AM, aschmid1 <[hidden email]> wrote: > Hi everyone, > I'm trying to estimate optimal Markowitz portfolio weights for a list of > stocks chosen upon some criterion using solve.QP from quadprog library. > When the number of stocks N reaches some limit, I get a message "matrix D > in quadratic function is not positive definite." For example, if I > rebalance every 6 weeks (which implies that variance is calculated for > 6-week interval prior to the period for which I calculate portfolio > weights), I can get solution for 25>=N<50. For 12-week interval, solution > exists for 50>=N<100, and for 24-week interval, I can get solution for > N=100. My attempt to remedy this problem with Higham's method doesn't help. > I'll greatly appreciate you input: first, why this may happen (can there be > lack of local minimum?), and second, whether there are R solvers that may > need only semi positive definite matrix. > > Thanks! Alec > > _______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance> -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions > should go. >         [[alternative HTML version deleted]] _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance-- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
Open this post in threaded view
|

## Re: Failure of solve.QP in portfolio modeling

 In reply to this post by Alec Schmidt-2 On Tue, 22 Sep 2015, aschmid1 <[hidden email]> writes: > Hi everyone, > I'm trying to estimate optimal Markowitz portfolio weights for a list > of stocks chosen upon some criterion using solve.QP from quadprog > library. When the number of stocks N reaches some limit, I get a > message "matrix D in quadratic function is not positive definite." For > example, if I rebalance every 6 weeks (which implies that variance is > calculated for 6-week interval prior to the period for which I > calculate portfolio weights), I can get solution for 25>=N<50. For > 12-week interval, solution exists for 50>=N<100, and for 24-week > interval, I can get solution for N=100. My attempt to remedy this > problem with Higham's method doesn't help. I'll greatly appreciate you > input: first, why this may happen (can there be lack of local > minimum?), and second, whether there are R solvers that may need only > semi positive definite matrix. > > Thanks! Alec > The thing you may want to look up is the "rank" of a matrix. For instance, I create a small data set R -- suppose these were daily-returns data of 10 equities.   na   <- 10 ## number of assets   nobs <- 10 ## number of observations   R <- array(rnorm(nobs * na, sd = 0.01), dim = c(nobs, na))   qr(cov(R))\$rank The rank of the covariance matrix is only 9; you need na+1 observations to get full rank. You can still compute the standard deviation of a portfolio:   ew <- rep(1/na, na) ## equal-weight portfolio   sqrt(ew %*% cov(R) %*% ew) But with a non-full rank matrix and no constraints, it is guaranteed that you have portfolios like this one:   zerovol <- svd(cov(R))\$v[,10]   sqrt(abs(zerovol %*% cov(R) %*% zerovol)) You get a zero-volatility portfolio. Whether that matters depends on your application. With constraints, perhaps not. An example in which it does not matter is in Section "1.3 Redundant assets" in https://cran.r-project.org/web/packages/NMOF/vignettes/TAportfolio.pdf  There you also have an example of a solver. A lengthier discussion is in Section "13.2.5 Repairing Matrices" in this book @BOOK{Gilli2011b,   title        = {Numerical Methods and Optimization in Finance},   publisher    = {Elsevier/Academic Press},   year         = 2011,   author       = {Gilli, Manfred and Maringer, Dietmar and Schumann,                   Enrico} } of which [disclosure], I am a co-author. -- Enrico Schumann Lucerne, Switzerland http://enricoschumann.net_______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance-- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
Open this post in threaded view
|

## Re: Failure of solve.QP in portfolio modeling

 In reply to this post by Alec Schmidt-2 You can use a factor model or shrinkage to get a positive definite variance matrix. There is a function for each in the BurStFin package on CRAN. The optimizer in Portfolio Probe doesn't care about positive definiteness (though that is not always a good thing).  It is free for academic use. Pat On 22/09/2015 14:37, aschmid1 wrote: > Hi everyone, > I'm trying to estimate optimal Markowitz portfolio weights for a list of > stocks chosen upon some criterion using solve.QP from quadprog library. > When the number of stocks N reaches some limit, I get a message "matrix > D in quadratic function is not positive definite." For example, if I > rebalance every 6 weeks (which implies that variance is calculated for > 6-week interval prior to the period for which I calculate portfolio > weights), I can get solution for 25>=N<50. For 12-week interval, > solution exists for 50>=N<100, and for 24-week interval, I can get > solution for N=100. My attempt to remedy this problem with Higham's > method doesn't help. I'll greatly appreciate you input: first, why this > may happen (can there be lack of local minimum?), and second, whether > there are R solvers that may need only semi positive definite matrix. > > Thanks! Alec > > _______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance> -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions > should go. -- Patrick Burns [hidden email] http://www.burns-stat.comhttp://www.portfolioprobe.com/blogtwitter: @burnsstat @portfolioprobe _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance-- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.