Hello everybody,
I have been using the random_portfolios function from the `PortfolioAnalytics` package to simulate the range of possibilities for return paths at each step under various portfolio constraints / mandates for evaluating mutual fund managers. As more managers are added to the universe, however, and more simulations are needed, the pure R implementations get pretty heavy and hard to scale. I was wondering if there has been any work out there thus far on implementing any of the three random portfolio generation methods (sample, simplex, and grid search) at a lower level, using something like `Rcpp` to enhance the efficiency of these algorithms? Any help/feedback is much appreciated. Thank you, -- Kshitij Dhingra Applied Academics LLC Office: +1.917.262.0516 Mobile: +1.206.696.5945 Email: [hidden email] Website: http://www.AppliedAcademics.com [[alternative HTML version deleted]] _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote:
> I have been using the random_portfolios function from the > `PortfolioAnalytics` package to simulate the range of possibilities > for return paths at each step under various portfolio constraints / > mandates for evaluating mutual fund managers. As more managers are > added to the universe, however, and more simulations are needed, the > pure R implementations get pretty heavy and hard to scale. I was > wondering if there has been any work out there thus far on > implementing any of the three random portfolio generation methods > (sample, simplex, and grid search) at a lower level, using something > like `Rcpp` to enhance the efficiency of these algorithms? We've discussed it, but I can't say that it is terribly high on our list of priorities. In most cases, no more than 1-2k portfolios should be required to get a fair view of the feasible space given your constraints and objectives. We'd be happy to work with you if you want to craft a patch to use C or Rcpp for this. Regards, Brian _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
Brian,
Thank you for a quick reply. I will soon be working on that problem and from what I have played with so far, it is unlikely that for our example ~2k portfolios will be enough (really hoping it would) to get a good sense of the feasible space and seems like I need to implement an Rcpp version of the random portfolios function. I will be happy to collaborate and share my code once i get a decent handle on it locally for the purposes of our current project. Regards, Kshitij Dhingra On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson <[hidden email]> wrote: > On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: > > I have been using the random_portfolios function from the > > `PortfolioAnalytics` package to simulate the range of possibilities > > for return paths at each step under various portfolio constraints / > > mandates for evaluating mutual fund managers. As more managers are > > added to the universe, however, and more simulations are needed, the > > pure R implementations get pretty heavy and hard to scale. I was > > wondering if there has been any work out there thus far on > > implementing any of the three random portfolio generation methods > > (sample, simplex, and grid search) at a lower level, using something > > like `Rcpp` to enhance the efficiency of these algorithms? > > We've discussed it, but I can't say that it is terribly high on our > list of priorities. > > In most cases, no more than 1-2k portfolios should be required to get a > fair view of the feasible space given your constraints and objectives. > > We'd be happy to work with you if you want to craft a patch to use C or > Rcpp for this. > > Regards, > > Brian > -- Kshitij Dhingra Applied Academics LLC Office: +1.917.262.0516 Mobile: +1.206.696.5945 Email: [hidden email] Website: http://www.AppliedAcademics.com [[alternative HTML version deleted]] _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
Kevin,
Can you give us a sense of the number of assets in the portfolio and the constraints? That will help us understand where the potential bottlenecks are in the random portfolio generation. For example, generating a set of random portfolios for box and weight constraints if relatively fast, but adding group or position limit constraints makes the algorithm more complicated and slower. Thanks, Ross On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra <[hidden email]> wrote: > Brian, > > Thank you for a quick reply. I will soon be working on that problem and > from what I have played with so far, it is unlikely that for our example > ~2k portfolios will be enough (really hoping it would) to get a good sense > of the feasible space and seems like I need to implement an Rcpp version of > the random portfolios function. I will be happy to collaborate and share my > code once i get a decent handle on it locally for the purposes of our > current project. > > Regards, > Kshitij Dhingra > > > > On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson <[hidden email]> > wrote: > >> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: >> > I have been using the random_portfolios function from the >> > `PortfolioAnalytics` package to simulate the range of possibilities >> > for return paths at each step under various portfolio constraints / >> > mandates for evaluating mutual fund managers. As more managers are >> > added to the universe, however, and more simulations are needed, the >> > pure R implementations get pretty heavy and hard to scale. I was >> > wondering if there has been any work out there thus far on >> > implementing any of the three random portfolio generation methods >> > (sample, simplex, and grid search) at a lower level, using something >> > like `Rcpp` to enhance the efficiency of these algorithms? >> >> We've discussed it, but I can't say that it is terribly high on our >> list of priorities. >> >> In most cases, no more than 1-2k portfolios should be required to get a >> fair view of the feasible space given your constraints and objectives. >> >> We'd be happy to work with you if you want to craft a patch to use C or >> Rcpp for this. >> >> Regards, >> >> Brian >> > > > > -- > Kshitij Dhingra > Applied Academics LLC > Office: +1.917.262.0516 > Mobile: +1.206.696.5945 > Email: [hidden email] > Website: http://www.AppliedAcademics.com > > [[alternative HTML version deleted]] > > _______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions should go. _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
Hi Ross,
Sure. Even though I have not profiled the bottlenecks quite in detail as of yet, i will give you a decent idea of the problem I am working with. I can have multiple indices with as much as 2000 assets with group, position and turnover limits (Not sure if i can increase the speed by removing constraints and doing rejection sampling later). In order to generate a daily possible set for the market in this case, I was playing around with ~4-5 thousand permutations. Also I think I will end up using the "sample" method because of the type of constraints we have and as you already have mentioned that method is the slowest (takes about 30 times the time using "simplex" for the same constraints). Adding box and position limit constraints are causing it to run a bit slower (but its not a big difference). I can always provide a more thorough analysis of the potential bottlenecks with a lot more detail when I have a chance to start working on translating it to cpp Thank you, On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett <[hidden email]> wrote: > Kevin, > > Can you give us a sense of the number of assets in the portfolio and > the constraints? That will help us understand where the potential > bottlenecks are in the random portfolio generation. For example, > generating a set of random portfolios for box and weight constraints > if relatively fast, but adding group or position limit constraints > makes the algorithm more complicated and slower. > > Thanks, > Ross > > > On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra > <[hidden email]> wrote: > > Brian, > > > > Thank you for a quick reply. I will soon be working on that problem and > > from what I have played with so far, it is unlikely that for our example > > ~2k portfolios will be enough (really hoping it would) to get a good > sense > > of the feasible space and seems like I need to implement an Rcpp version > of > > the random portfolios function. I will be happy to collaborate and share > my > > code once i get a decent handle on it locally for the purposes of our > > current project. > > > > Regards, > > Kshitij Dhingra > > > > > > > > On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson <[hidden email]> > > wrote: > > > >> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: > >> > I have been using the random_portfolios function from the > >> > `PortfolioAnalytics` package to simulate the range of possibilities > >> > for return paths at each step under various portfolio constraints / > >> > mandates for evaluating mutual fund managers. As more managers are > >> > added to the universe, however, and more simulations are needed, the > >> > pure R implementations get pretty heavy and hard to scale. I was > >> > wondering if there has been any work out there thus far on > >> > implementing any of the three random portfolio generation methods > >> > (sample, simplex, and grid search) at a lower level, using something > >> > like `Rcpp` to enhance the efficiency of these algorithms? > >> > >> We've discussed it, but I can't say that it is terribly high on our > >> list of priorities. > >> > >> In most cases, no more than 1-2k portfolios should be required to get a > >> fair view of the feasible space given your constraints and objectives. > >> > >> We'd be happy to work with you if you want to craft a patch to use C or > >> Rcpp for this. > >> > >> Regards, > >> > >> Brian > >> > > > > > > > > -- > > Kshitij Dhingra > > Applied Academics LLC > > Office: +1.917.262.0516 > > Mobile: +1.206.696.5945 > > Email: [hidden email] > > Website: http://www.AppliedAcademics.com > > > > [[alternative HTML version deleted]] > > > > _______________________________________________ > > [hidden email] mailing list > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > > -- Subscriber-posting only. If you want to post, subscribe first. > > -- Also note that this is not the r-help list where general R questions > should go. > -- Kshitij Dhingra Applied Academics LLC Office: +1.917.262.0516 Mobile: +1.206.696.5945 Email: [hidden email] Website: http://www.AppliedAcademics.com [[alternative HTML version deleted]] _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
For this type of problem, I would probably generate one set of random
portfolios and just reuse that set of feasible portfolios... My usual rule is n-assets + 1-2k feasible portfolios. You can get a better number e.g. from sampling theory, but this should be enough. Once you have this weights matrix rp, you only need to regenerate rp if your universe changes. Still interested in a more efficient implementation, of course, or we can work with you to see if we can find resources to work on it, e.g. from academia. Regards, Brian On 03/20/2017 05:28 PM, Kevin Dhingra wrote: > Hi Ross, > > Sure. Even though I have not profiled the bottlenecks quite in detail as of > yet, i will give you a decent idea of the problem I am working with. I can > have multiple indices with as much as 2000 assets with group, position and > turnover limits (Not sure if i can increase the speed by removing > constraints and doing rejection sampling later). In order to generate a > daily possible set for the market in this case, I was playing around with > ~4-5 thousand permutations. Also I think I will end up using the "sample" > method because of the type of constraints we have and as you already have > mentioned that method is the slowest (takes about 30 times the time using > "simplex" for the same constraints). Adding box and position limit > constraints are causing it to run a bit slower (but its not a big > difference). I can always provide a more thorough analysis of the potential > bottlenecks with a lot more detail when I have a chance to start working on > translating it to cpp > > Thank you, > > On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett <[hidden email]> > wrote: > >> Kevin, >> >> Can you give us a sense of the number of assets in the portfolio and >> the constraints? That will help us understand where the potential >> bottlenecks are in the random portfolio generation. For example, >> generating a set of random portfolios for box and weight constraints >> if relatively fast, but adding group or position limit constraints >> makes the algorithm more complicated and slower. >> >> Thanks, >> Ross >> >> >> On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra >> <[hidden email]> wrote: >>> Brian, >>> >>> Thank you for a quick reply. I will soon be working on that problem and >>> from what I have played with so far, it is unlikely that for our example >>> ~2k portfolios will be enough (really hoping it would) to get a good >> sense >>> of the feasible space and seems like I need to implement an Rcpp version >> of >>> the random portfolios function. I will be happy to collaborate and share >> my >>> code once i get a decent handle on it locally for the purposes of our >>> current project. >>> >>> Regards, >>> Kshitij Dhingra >>> >>> >>> >>> On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson <[hidden email]> >>> wrote: >>> >>>> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: >>>>> I have been using the random_portfolios function from the >>>>> `PortfolioAnalytics` package to simulate the range of possibilities >>>>> for return paths at each step under various portfolio constraints / >>>>> mandates for evaluating mutual fund managers. As more managers are >>>>> added to the universe, however, and more simulations are needed, the >>>>> pure R implementations get pretty heavy and hard to scale. I was >>>>> wondering if there has been any work out there thus far on >>>>> implementing any of the three random portfolio generation methods >>>>> (sample, simplex, and grid search) at a lower level, using something >>>>> like `Rcpp` to enhance the efficiency of these algorithms? >>>> >>>> We've discussed it, but I can't say that it is terribly high on our >>>> list of priorities. >>>> >>>> In most cases, no more than 1-2k portfolios should be required to get a >>>> fair view of the feasible space given your constraints and objectives. >>>> >>>> We'd be happy to work with you if you want to craft a patch to use C or >>>> Rcpp for this. >>>> >>>> Regards, >>>> >>>> Brian >>>> >>> >>> >>> >>> -- >>> Kshitij Dhingra >>> Applied Academics LLC >>> Office: +1.917.262.0516 >>> Mobile: +1.206.696.5945 >>> Email: [hidden email] >>> Website: http://www.AppliedAcademics.com >>> >>> [[alternative HTML version deleted]] >>> >>> _______________________________________________ >>> [hidden email] mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance >>> -- Subscriber-posting only. If you want to post, subscribe first. >>> -- Also note that this is not the r-help list where general R questions >> should go. >> > > > -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
Brian,
Yes I think that will be a good starting point. My universe would not change a lot (I will be working with 10-15 benchmarks at a time and I guess I can generate a reusable set for each independently before running it through my main algorithm). Having said that, I envision the investment mandates/constraints changing quite a lot (both in the cross section and also over time for the same manager). I am hoping there must be a way around it using rejection sampling but have not done enough research to comment on how that solution works for such big dimensions. It will be really helpful if you could point me to any specific resources from academia for the same (Haven't been able to find much about random portfolios myself except Portfolio Analytics and Patrick Burns work on Portfolio Probe). As a side note - Do you think translating it using Rcpp would be time well spent or you think there must be a smarter way to get around it still using R? I really appreciate your help on this thread. Regards, Kshitij Dhingra On Mon, Mar 20, 2017 at 7:21 PM, Brian G. Peterson <[hidden email]> wrote: > For this type of problem, I would probably generate one set of random > portfolios and just reuse that set of feasible portfolios... My usual rule > is n-assets + 1-2k feasible portfolios. You can get a better number e.g. > from sampling theory, but this should be enough. > > Once you have this weights matrix rp, you only need to regenerate rp if > your universe changes. > > Still interested in a more efficient implementation, of course, or we can > work with you to see if we can find resources to work on it, e.g. from > academia. > > Regards, > > Brian > > > On 03/20/2017 05:28 PM, Kevin Dhingra wrote: > >> Hi Ross, >> >> Sure. Even though I have not profiled the bottlenecks quite in detail as >> of >> yet, i will give you a decent idea of the problem I am working with. I can >> have multiple indices with as much as 2000 assets with group, position and >> turnover limits (Not sure if i can increase the speed by removing >> constraints and doing rejection sampling later). In order to generate a >> daily possible set for the market in this case, I was playing around with >> ~4-5 thousand permutations. Also I think I will end up using the "sample" >> method because of the type of constraints we have and as you already have >> mentioned that method is the slowest (takes about 30 times the time using >> "simplex" for the same constraints). Adding box and position limit >> constraints are causing it to run a bit slower (but its not a big >> difference). I can always provide a more thorough analysis of the >> potential >> bottlenecks with a lot more detail when I have a chance to start working >> on >> translating it to cpp >> >> Thank you, >> >> On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett <[hidden email]> >> wrote: >> >> Kevin, >>> >>> Can you give us a sense of the number of assets in the portfolio and >>> the constraints? That will help us understand where the potential >>> bottlenecks are in the random portfolio generation. For example, >>> generating a set of random portfolios for box and weight constraints >>> if relatively fast, but adding group or position limit constraints >>> makes the algorithm more complicated and slower. >>> >>> Thanks, >>> Ross >>> >>> >>> On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra >>> <[hidden email]> wrote: >>> >>>> Brian, >>>> >>>> Thank you for a quick reply. I will soon be working on that problem and >>>> from what I have played with so far, it is unlikely that for our example >>>> ~2k portfolios will be enough (really hoping it would) to get a good >>>> >>> sense >>> >>>> of the feasible space and seems like I need to implement an Rcpp version >>>> >>> of >>> >>>> the random portfolios function. I will be happy to collaborate and share >>>> >>> my >>> >>>> code once i get a decent handle on it locally for the purposes of our >>>> current project. >>>> >>>> Regards, >>>> Kshitij Dhingra >>>> >>>> >>>> >>>> On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson <[hidden email] >>>> > >>>> wrote: >>>> >>>> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: >>>>> >>>>>> I have been using the random_portfolios function from the >>>>>> `PortfolioAnalytics` package to simulate the range of possibilities >>>>>> for return paths at each step under various portfolio constraints / >>>>>> mandates for evaluating mutual fund managers. As more managers are >>>>>> added to the universe, however, and more simulations are needed, the >>>>>> pure R implementations get pretty heavy and hard to scale. I was >>>>>> wondering if there has been any work out there thus far on >>>>>> implementing any of the three random portfolio generation methods >>>>>> (sample, simplex, and grid search) at a lower level, using something >>>>>> like `Rcpp` to enhance the efficiency of these algorithms? >>>>>> >>>>> >>>>> We've discussed it, but I can't say that it is terribly high on our >>>>> list of priorities. >>>>> >>>>> In most cases, no more than 1-2k portfolios should be required to get a >>>>> fair view of the feasible space given your constraints and objectives. >>>>> >>>>> We'd be happy to work with you if you want to craft a patch to use C or >>>>> Rcpp for this. >>>>> >>>>> Regards, >>>>> >>>>> Brian >>>>> >>>>> >>>> >>>> >>>> -- >>>> Kshitij Dhingra >>>> Applied Academics LLC >>>> Office: +1.917.262.0516 >>>> Mobile: +1.206.696.5945 >>>> Email: [hidden email] >>>> Website: http://www.AppliedAcademics.com >>>> >>>> [[alternative HTML version deleted]] >>>> >>>> _______________________________________________ >>>> [hidden email] mailing list >>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance >>>> -- Subscriber-posting only. If you want to post, subscribe first. >>>> -- Also note that this is not the r-help list where general R questions >>>> >>> should go. >>> >>> >> >> >> > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > > > _______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions > should go. > -- Kshitij Dhingra Applied Academics LLC Office: +1.917.262.0516 Mobile: +1.206.696.5945 Email: [hidden email] Website: http://www.AppliedAcademics.com [[alternative HTML version deleted]] _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
In reply to this post by Kevin Dhingra
Please look at my package rportfolios. It allows you to construct individual and samples of random portfolios that satisfy a wide range of constraints ( e.g., long only, short only, long short portfolios with notional exposure constraints, bounded weights).
Regards, Frederick Novomestky -----Original Message----- >From: Kevin Dhingra <[hidden email]> >Sent: Mar 20, 2017 6:28 PM >To: Ross Bennett <[hidden email]> >Cc: "[hidden email]" <[hidden email]> >Subject: Re: [R-SIG-Finance] random portfolios > >Hi Ross, > >Sure. Even though I have not profiled the bottlenecks quite in detail as of >yet, i will give you a decent idea of the problem I am working with. I can >have multiple indices with as much as 2000 assets with group, position and >turnover limits (Not sure if i can increase the speed by removing >constraints and doing rejection sampling later). In order to generate a >daily possible set for the market in this case, I was playing around with >~4-5 thousand permutations. Also I think I will end up using the "sample" >method because of the type of constraints we have and as you already have >mentioned that method is the slowest (takes about 30 times the time using >"simplex" for the same constraints). Adding box and position limit >constraints are causing it to run a bit slower (but its not a big >difference). I can always provide a more thorough analysis of the potential >bottlenecks with a lot more detail when I have a chance to start working on >translating it to cpp > >Thank you, > >On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett <[hidden email]> >wrote: > >> Kevin, >> >> Can you give us a sense of the number of assets in the portfolio and >> the constraints? That will help us understand where the potential >> bottlenecks are in the random portfolio generation. For example, >> generating a set of random portfolios for box and weight constraints >> if relatively fast, but adding group or position limit constraints >> makes the algorithm more complicated and slower. >> >> Thanks, >> Ross >> >> >> On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra >> <[hidden email]> wrote: >> > Brian, >> > >> > Thank you for a quick reply. I will soon be working on that problem and >> > from what I have played with so far, it is unlikely that for our example >> > ~2k portfolios will be enough (really hoping it would) to get a good >> sense >> > of the feasible space and seems like I need to implement an Rcpp version >> of >> > the random portfolios function. I will be happy to collaborate and share >> my >> > code once i get a decent handle on it locally for the purposes of our >> > current project. >> > >> > Regards, >> > Kshitij Dhingra >> > >> > >> > >> > On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson <[hidden email]> >> > wrote: >> > >> >> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: >> >> > I have been using the random_portfolios function from the >> >> > `PortfolioAnalytics` package to simulate the range of possibilities >> >> > for return paths at each step under various portfolio constraints / >> >> > mandates for evaluating mutual fund managers. As more managers are >> >> > added to the universe, however, and more simulations are needed, the >> >> > pure R implementations get pretty heavy and hard to scale. I was >> >> > wondering if there has been any work out there thus far on >> >> > implementing any of the three random portfolio generation methods >> >> > (sample, simplex, and grid search) at a lower level, using something >> >> > like `Rcpp` to enhance the efficiency of these algorithms? >> >> >> >> We've discussed it, but I can't say that it is terribly high on our >> >> list of priorities. >> >> >> >> In most cases, no more than 1-2k portfolios should be required to get a >> >> fair view of the feasible space given your constraints and objectives. >> >> >> >> We'd be happy to work with you if you want to craft a patch to use C or >> >> Rcpp for this. >> >> >> >> Regards, >> >> >> >> Brian >> >> >> > >> > >> > >> > -- >> > Kshitij Dhingra >> > Applied Academics LLC >> > Office: +1.917.262.0516 >> > Mobile: +1.206.696.5945 >> > Email: [hidden email] >> > Website: http://www.AppliedAcademics.com >> > >> > [[alternative HTML version deleted]] >> > >> > _______________________________________________ >> > [hidden email] mailing list >> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance >> > -- Subscriber-posting only. If you want to post, subscribe first. >> > -- Also note that this is not the r-help list where general R questions >> should go. >> > > > >-- >Kshitij Dhingra >Applied Academics LLC >Office: +1.917.262.0516 >Mobile: +1.206.696.5945 >Email: [hidden email] >Website: http://www.AppliedAcademics.com > > [[alternative HTML version deleted]] > >_______________________________________________ >[hidden email] mailing list >https://stat.ethz.ch/mailman/listinfo/r-sig-finance >-- Subscriber-posting only. If you want to post, subscribe first. >-- Also note that this is not the r-help list where general R questions should go. _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
In reply to this post by Kevin Dhingra
The process you describe is pretty standard for an investment-committee
driven process. I'm going to suggest that you don't really want to change the constraints that often. For example, box constraints should be as large as your overall investment mandate allows to give you the greatest possible room for allocations. Sector or Factor constraints likewise should be as minimal as possible just to guarantee the degree of diversification described in your investment mandate. The reason I'm suggesting this minimal constraint set is one of the reasons we wrote the random portfolio code in the first place. To see what I mean, generate a set of unconstrained random portfolios (or e.g. only with a full-investment constraint). Then generate sets of constrained random portfolios, adding your various constraint sets. Plot the different sets on the same risk/return scatter plot, using different colors for each set. Note how small the feasible space becomes, very quickly. This shrinkage of the feasible space has some good shrinkage properties... moderate shrinkage actually decreases the possible impact of estimation error in the various inputs, a little. Large amounts of shrinkage (overly restrictive constraints) will do the opposite, and magnify the negative out of sample impact of estimation error. The academic literature mostly focuses on analytical solvers (e.g. quadratic, linear, etc) and simple constraint sets. We've cited papers by Patrick Burns as well as papers on the simplex models in PortfolioAnalytics, but the literature is not vast. Numerical solvers become important as the feasible space becomes non-smooth. One of the things that can create a non-smooth feasible space is a complex, overlapping constraint set. The rportfolios package proposed by Frederick Novomestky also seems to be an R-only implementation, at a glance relying on truncated random binomial vectors rather than truncated random uniform vectors. I believe it will have similar performance characteristics to the Burns-style random sample portfolios, and it seems to support fewer constraint sets (no overlapping sector, group, or factor constraints that I see). In any case, it generates matrices of weights that are likely compatible with the PortfolioAnalytics random or seed portfolio inputs. So if it works for you, that's great. You also discuss using rejection after generating the portfolios. This is the method used internally by random.portfolios to reject individual weights if a constraint is violated. I'll have to evaluate whether the truncdist package used by rportfolios could be more efficient than the runif that is used by the current code. PortfolioAnalytics also allows portfolios to be penalized in the solver, so that more complex cases can be considered, or interactions between constraints and objectives. To answer the question of whether Rcpp will help is somewhat complex. I'm confident that some of the nested loops in the generation code will be sped up by Rcpp. It is possible that more efficient algorithms are available for constructing the weight vectors. A reason that this hasn't been a huge priority though is that construction of the random portfolio matrix is usually not the time limiter in a large optimization: your objective function is. I think it will be possible to improve the efficiency of this step, though it is unclear how much of an impact this should have in practice to a large and complicated numerically solved portfolio optimization problem. Regards, Brian On 03/20/2017 07:06 PM, Kevin Dhingra wrote: > Brian, > > Yes I think that will be a good starting point. My universe would not > change a lot (I will be working with 10-15 benchmarks at a time and I guess > I can generate a reusable set for each independently before running it > through my main algorithm). Having said that, I envision the investment > mandates/constraints changing quite a lot (both in the cross section and > also over time for the same manager). I am hoping there must be a way > around it using rejection sampling but have not done enough research to > comment on how that solution works for such big dimensions. It will be > really helpful if you could point me to any specific resources from > academia for the same (Haven't been able to find much about random > portfolios myself except Portfolio Analytics and Patrick Burns work on > Portfolio Probe). As a side note - Do you think translating it using Rcpp > would be time well spent or you think there must be a smarter way to get > around it still using R? > > I really appreciate your help on this thread. > > Regards, > Kshitij Dhingra > > On Mon, Mar 20, 2017 at 7:21 PM, Brian G. Peterson <[hidden email]> > wrote: > >> For this type of problem, I would probably generate one set of random >> portfolios and just reuse that set of feasible portfolios... My usual rule >> is n-assets + 1-2k feasible portfolios. You can get a better number e.g. >> from sampling theory, but this should be enough. >> >> Once you have this weights matrix rp, you only need to regenerate rp if >> your universe changes. >> >> Still interested in a more efficient implementation, of course, or we can >> work with you to see if we can find resources to work on it, e.g. from >> academia. >> >> Regards, >> >> Brian >> >> >> On 03/20/2017 05:28 PM, Kevin Dhingra wrote: >> >>> Hi Ross, >>> >>> Sure. Even though I have not profiled the bottlenecks quite in detail as >>> of >>> yet, i will give you a decent idea of the problem I am working with. I can >>> have multiple indices with as much as 2000 assets with group, position and >>> turnover limits (Not sure if i can increase the speed by removing >>> constraints and doing rejection sampling later). In order to generate a >>> daily possible set for the market in this case, I was playing around with >>> ~4-5 thousand permutations. Also I think I will end up using the "sample" >>> method because of the type of constraints we have and as you already have >>> mentioned that method is the slowest (takes about 30 times the time using >>> "simplex" for the same constraints). Adding box and position limit >>> constraints are causing it to run a bit slower (but its not a big >>> difference). I can always provide a more thorough analysis of the >>> potential >>> bottlenecks with a lot more detail when I have a chance to start working >>> on >>> translating it to cpp >>> >>> Thank you, >>> >>> On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett <[hidden email]> >>> wrote: >>> >>> Kevin, >>>> >>>> Can you give us a sense of the number of assets in the portfolio and >>>> the constraints? That will help us understand where the potential >>>> bottlenecks are in the random portfolio generation. For example, >>>> generating a set of random portfolios for box and weight constraints >>>> if relatively fast, but adding group or position limit constraints >>>> makes the algorithm more complicated and slower. >>>> >>>> Thanks, >>>> Ross >>>> >>>> >>>> On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra >>>> <[hidden email]> wrote: >>>> >>>>> Brian, >>>>> >>>>> Thank you for a quick reply. I will soon be working on that problem and >>>>> from what I have played with so far, it is unlikely that for our example >>>>> ~2k portfolios will be enough (really hoping it would) to get a good >>>>> >>>> sense >>>> >>>>> of the feasible space and seems like I need to implement an Rcpp version >>>>> >>>> of >>>> >>>>> the random portfolios function. I will be happy to collaborate and share >>>>> >>>> my >>>> >>>>> code once i get a decent handle on it locally for the purposes of our >>>>> current project. >>>>> >>>>> Regards, >>>>> Kshitij Dhingra >>>>> >>>>> >>>>> >>>>> On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson <[hidden email] >>>>>> >>>>> wrote: >>>>> >>>>> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: >>>>>> >>>>>>> I have been using the random_portfolios function from the >>>>>>> `PortfolioAnalytics` package to simulate the range of possibilities >>>>>>> for return paths at each step under various portfolio constraints / >>>>>>> mandates for evaluating mutual fund managers. As more managers are >>>>>>> added to the universe, however, and more simulations are needed, the >>>>>>> pure R implementations get pretty heavy and hard to scale. I was >>>>>>> wondering if there has been any work out there thus far on >>>>>>> implementing any of the three random portfolio generation methods >>>>>>> (sample, simplex, and grid search) at a lower level, using something >>>>>>> like `Rcpp` to enhance the efficiency of these algorithms? >>>>>>> >>>>>> >>>>>> We've discussed it, but I can't say that it is terribly high on our >>>>>> list of priorities. >>>>>> >>>>>> In most cases, no more than 1-2k portfolios should be required to get a >>>>>> fair view of the feasible space given your constraints and objectives. >>>>>> >>>>>> We'd be happy to work with you if you want to craft a patch to use C or >>>>>> Rcpp for this. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Brian >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Kshitij Dhingra >>>>> Applied Academics LLC >>>>> Office: +1.917.262.0516 >>>>> Mobile: +1.206.696.5945 >>>>> Email: [hidden email] >>>>> Website: http://www.AppliedAcademics.com >>>>> >>>>> [[alternative HTML version deleted]] >>>>> >>>>> _______________________________________________ >>>>> [hidden email] mailing list >>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance >>>>> -- Subscriber-posting only. If you want to post, subscribe first. >>>>> -- Also note that this is not the r-help list where general R questions >>>>> >>>> should go. >>>> >>>> >>> >>> >>> >> >> -- >> Brian G. Peterson >> http://braverock.com/brian/ >> Ph: 773-459-4973 >> IM: bgpbraverock >> >> >> _______________________________________________ >> [hidden email] mailing list >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance >> -- Subscriber-posting only. If you want to post, subscribe first. >> -- Also note that this is not the r-help list where general R questions >> should go. >> > > > -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
Just wanted to point out an example of what Brian mentioned for
visualizing the feasible space with different constraint sets. Visualizations here http://rossb34.github.io/PortfolioAnalyticsPresentation2016/#22 on slides 22-26 and code https://github.com/rossb34/PortfolioAnalyticsPresentation2016/blob/master/feasible_space.R to produce the plots. Ross On Tue, Mar 21, 2017 at 5:41 AM, Brian G. Peterson <[hidden email]> wrote: > The process you describe is pretty standard for an investment-committee > driven process. > > I'm going to suggest that you don't really want to change the constraints > that often. For example, box constraints should be as large as your overall > investment mandate allows to give you the greatest possible room for > allocations. Sector or Factor constraints likewise should be as minimal as > possible just to guarantee the degree of diversification described in your > investment mandate. > > The reason I'm suggesting this minimal constraint set is one of the reasons > we wrote the random portfolio code in the first place. To see what I mean, > generate a set of unconstrained random portfolios (or e.g. only with a > full-investment constraint). Then generate sets of constrained random > portfolios, adding your various constraint sets. Plot the different sets on > the same risk/return scatter plot, using different colors for each set. > Note how small the feasible space becomes, very quickly. > > This shrinkage of the feasible space has some good shrinkage properties... > moderate shrinkage actually decreases the possible impact of estimation > error in the various inputs, a little. Large amounts of shrinkage (overly > restrictive constraints) will do the opposite, and magnify the negative out > of sample impact of estimation error. > > The academic literature mostly focuses on analytical solvers (e.g. > quadratic, linear, etc) and simple constraint sets. We've cited papers by > Patrick Burns as well as papers on the simplex models in PortfolioAnalytics, > but the literature is not vast. > > Numerical solvers become important as the feasible space becomes non-smooth. > One of the things that can create a non-smooth feasible space is a complex, > overlapping constraint set. > > The rportfolios package proposed by Frederick Novomestky also seems to be an > R-only implementation, at a glance relying on truncated random binomial > vectors rather than truncated random uniform vectors. I believe it will > have similar performance characteristics to the Burns-style random sample > portfolios, and it seems to support fewer constraint sets (no overlapping > sector, group, or factor constraints that I see). In any case, it generates > matrices of weights that are likely compatible with the PortfolioAnalytics > random or seed portfolio inputs. So if it works for you, that's great. > > You also discuss using rejection after generating the portfolios. This is > the method used internally by random.portfolios to reject individual weights > if a constraint is violated. I'll have to evaluate whether the truncdist > package used by rportfolios could be more efficient than the runif that is > used by the current code. PortfolioAnalytics also allows portfolios to be > penalized in the solver, so that more complex cases can be considered, or > interactions between constraints and objectives. > > To answer the question of whether Rcpp will help is somewhat complex. I'm > confident that some of the nested loops in the generation code will be sped > up by Rcpp. It is possible that more efficient algorithms are available for > constructing the weight vectors. A reason that this hasn't been a huge > priority though is that construction of the random portfolio matrix is > usually not the time limiter in a large optimization: your objective > function is. I think it will be possible to improve the efficiency of this > step, though it is unclear how much of an impact this should have in > practice to a large and complicated numerically solved portfolio > optimization problem. > > Regards, > > Brian > > > > > On 03/20/2017 07:06 PM, Kevin Dhingra wrote: >> >> Brian, >> >> Yes I think that will be a good starting point. My universe would not >> change a lot (I will be working with 10-15 benchmarks at a time and I >> guess >> I can generate a reusable set for each independently before running it >> through my main algorithm). Having said that, I envision the investment >> mandates/constraints changing quite a lot (both in the cross section and >> also over time for the same manager). I am hoping there must be a way >> around it using rejection sampling but have not done enough research to >> comment on how that solution works for such big dimensions. It will be >> really helpful if you could point me to any specific resources from >> academia for the same (Haven't been able to find much about random >> portfolios myself except Portfolio Analytics and Patrick Burns work on >> Portfolio Probe). As a side note - Do you think translating it using Rcpp >> would be time well spent or you think there must be a smarter way to get >> around it still using R? >> >> I really appreciate your help on this thread. >> >> Regards, >> Kshitij Dhingra >> >> On Mon, Mar 20, 2017 at 7:21 PM, Brian G. Peterson <[hidden email]> >> wrote: >> >>> For this type of problem, I would probably generate one set of random >>> portfolios and just reuse that set of feasible portfolios... My usual >>> rule >>> is n-assets + 1-2k feasible portfolios. You can get a better number e.g. >>> from sampling theory, but this should be enough. >>> >>> Once you have this weights matrix rp, you only need to regenerate rp if >>> your universe changes. >>> >>> Still interested in a more efficient implementation, of course, or we can >>> work with you to see if we can find resources to work on it, e.g. from >>> academia. >>> >>> Regards, >>> >>> Brian >>> >>> >>> On 03/20/2017 05:28 PM, Kevin Dhingra wrote: >>> >>>> Hi Ross, >>>> >>>> Sure. Even though I have not profiled the bottlenecks quite in detail as >>>> of >>>> yet, i will give you a decent idea of the problem I am working with. I >>>> can >>>> have multiple indices with as much as 2000 assets with group, position >>>> and >>>> turnover limits (Not sure if i can increase the speed by removing >>>> constraints and doing rejection sampling later). In order to generate a >>>> daily possible set for the market in this case, I was playing around >>>> with >>>> ~4-5 thousand permutations. Also I think I will end up using the >>>> "sample" >>>> method because of the type of constraints we have and as you already >>>> have >>>> mentioned that method is the slowest (takes about 30 times the time >>>> using >>>> "simplex" for the same constraints). Adding box and position limit >>>> constraints are causing it to run a bit slower (but its not a big >>>> difference). I can always provide a more thorough analysis of the >>>> potential >>>> bottlenecks with a lot more detail when I have a chance to start working >>>> on >>>> translating it to cpp >>>> >>>> Thank you, >>>> >>>> On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett <[hidden email]> >>>> wrote: >>>> >>>> Kevin, >>>>> >>>>> >>>>> Can you give us a sense of the number of assets in the portfolio and >>>>> the constraints? That will help us understand where the potential >>>>> bottlenecks are in the random portfolio generation. For example, >>>>> generating a set of random portfolios for box and weight constraints >>>>> if relatively fast, but adding group or position limit constraints >>>>> makes the algorithm more complicated and slower. >>>>> >>>>> Thanks, >>>>> Ross >>>>> >>>>> >>>>> On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra >>>>> <[hidden email]> wrote: >>>>> >>>>>> Brian, >>>>>> >>>>>> Thank you for a quick reply. I will soon be working on that problem >>>>>> and >>>>>> from what I have played with so far, it is unlikely that for our >>>>>> example >>>>>> ~2k portfolios will be enough (really hoping it would) to get a good >>>>>> >>>>> sense >>>>> >>>>>> of the feasible space and seems like I need to implement an Rcpp >>>>>> version >>>>>> >>>>> of >>>>> >>>>>> the random portfolios function. I will be happy to collaborate and >>>>>> share >>>>>> >>>>> my >>>>> >>>>>> code once i get a decent handle on it locally for the purposes of our >>>>>> current project. >>>>>> >>>>>> Regards, >>>>>> Kshitij Dhingra >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson >>>>>> <[hidden email] >>>>>>> >>>>>>> >>>>>> wrote: >>>>>> >>>>>> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: >>>>>>> >>>>>>> >>>>>>>> I have been using the random_portfolios function from the >>>>>>>> `PortfolioAnalytics` package to simulate the range of possibilities >>>>>>>> for return paths at each step under various portfolio constraints / >>>>>>>> mandates for evaluating mutual fund managers. As more managers are >>>>>>>> added to the universe, however, and more simulations are needed, the >>>>>>>> pure R implementations get pretty heavy and hard to scale. I was >>>>>>>> wondering if there has been any work out there thus far on >>>>>>>> implementing any of the three random portfolio generation methods >>>>>>>> (sample, simplex, and grid search) at a lower level, using something >>>>>>>> like `Rcpp` to enhance the efficiency of these algorithms? >>>>>>>> >>>>>>> >>>>>>> We've discussed it, but I can't say that it is terribly high on our >>>>>>> list of priorities. >>>>>>> >>>>>>> In most cases, no more than 1-2k portfolios should be required to get >>>>>>> a >>>>>>> fair view of the feasible space given your constraints and >>>>>>> objectives. >>>>>>> >>>>>>> We'd be happy to work with you if you want to craft a patch to use C >>>>>>> or >>>>>>> Rcpp for this. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Brian >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Kshitij Dhingra >>>>>> Applied Academics LLC >>>>>> Office: +1.917.262.0516 >>>>>> Mobile: +1.206.696.5945 >>>>>> Email: [hidden email] >>>>>> Website: http://www.AppliedAcademics.com >>>>>> >>>>>> [[alternative HTML version deleted]] >>>>>> >>>>>> _______________________________________________ >>>>>> [hidden email] mailing list >>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance >>>>>> -- Subscriber-posting only. If you want to post, subscribe first. >>>>>> -- Also note that this is not the r-help list where general R >>>>>> questions >>>>>> >>>>> should go. >>>>> >>>>> >>>> >>>> >>>> >>> >>> -- >>> Brian G. Peterson >>> http://braverock.com/brian/ >>> Ph: 773-459-4973 >>> IM: bgpbraverock >>> >>> >>> _______________________________________________ >>> [hidden email] mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance >>> -- Subscriber-posting only. If you want to post, subscribe first. >>> -- Also note that this is not the r-help list where general R questions >>> should go. >>> >> >> >> > > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > > _______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions > should go. _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
Brian, Ross and Frednovo,
This has been extremely helpful. I think now I have a lot better understanding of how to think about the problem at hand. Appreciate it. Regards, Kshitij Dhingra On Tue, Mar 21, 2017 at 7:24 AM, Ross Bennett <[hidden email]> wrote: > Just wanted to point out an example of what Brian mentioned for > visualizing the feasible space with different constraint sets. > Visualizations here > http://rossb34.github.io/PortfolioAnalyticsPresentation2016/#22 on > slides 22-26 and code > https://github.com/rossb34/PortfolioAnalyticsPresentation > 2016/blob/master/feasible_space.R > to produce the plots. > > Ross > > > On Tue, Mar 21, 2017 at 5:41 AM, Brian G. Peterson <[hidden email]> > wrote: > > The process you describe is pretty standard for an investment-committee > > driven process. > > > > I'm going to suggest that you don't really want to change the constraints > > that often. For example, box constraints should be as large as your > overall > > investment mandate allows to give you the greatest possible room for > > allocations. Sector or Factor constraints likewise should be as minimal > as > > possible just to guarantee the degree of diversification described in > your > > investment mandate. > > > > The reason I'm suggesting this minimal constraint set is one of the > reasons > > we wrote the random portfolio code in the first place. To see what I > mean, > > generate a set of unconstrained random portfolios (or e.g. only with a > > full-investment constraint). Then generate sets of constrained random > > portfolios, adding your various constraint sets. Plot the different sets > on > > the same risk/return scatter plot, using different colors for each set. > > Note how small the feasible space becomes, very quickly. > > > > This shrinkage of the feasible space has some good shrinkage > properties... > > moderate shrinkage actually decreases the possible impact of estimation > > error in the various inputs, a little. Large amounts of shrinkage > (overly > > restrictive constraints) will do the opposite, and magnify the negative > out > > of sample impact of estimation error. > > > > The academic literature mostly focuses on analytical solvers (e.g. > > quadratic, linear, etc) and simple constraint sets. We've cited papers > by > > Patrick Burns as well as papers on the simplex models in > PortfolioAnalytics, > > but the literature is not vast. > > > > Numerical solvers become important as the feasible space becomes > non-smooth. > > One of the things that can create a non-smooth feasible space is a > complex, > > overlapping constraint set. > > > > The rportfolios package proposed by Frederick Novomestky also seems to > be an > > R-only implementation, at a glance relying on truncated random binomial > > vectors rather than truncated random uniform vectors. I believe it will > > have similar performance characteristics to the Burns-style random sample > > portfolios, and it seems to support fewer constraint sets (no overlapping > > sector, group, or factor constraints that I see). In any case, it > generates > > matrices of weights that are likely compatible with the > PortfolioAnalytics > > random or seed portfolio inputs. So if it works for you, that's great. > > > > You also discuss using rejection after generating the portfolios. This > is > > the method used internally by random.portfolios to reject individual > weights > > if a constraint is violated. I'll have to evaluate whether the truncdist > > package used by rportfolios could be more efficient than the runif that > is > > used by the current code. PortfolioAnalytics also allows portfolios to > be > > penalized in the solver, so that more complex cases can be considered, or > > interactions between constraints and objectives. > > > > To answer the question of whether Rcpp will help is somewhat complex. I'm > > confident that some of the nested loops in the generation code will be > sped > > up by Rcpp. It is possible that more efficient algorithms are available > for > > constructing the weight vectors. A reason that this hasn't been a huge > > priority though is that construction of the random portfolio matrix is > > usually not the time limiter in a large optimization: your objective > > function is. I think it will be possible to improve the efficiency of > this > > step, though it is unclear how much of an impact this should have in > > practice to a large and complicated numerically solved portfolio > > optimization problem. > > > > Regards, > > > > Brian > > > > > > > > > > On 03/20/2017 07:06 PM, Kevin Dhingra wrote: > >> > >> Brian, > >> > >> Yes I think that will be a good starting point. My universe would not > >> change a lot (I will be working with 10-15 benchmarks at a time and I > >> guess > >> I can generate a reusable set for each independently before running it > >> through my main algorithm). Having said that, I envision the investment > >> mandates/constraints changing quite a lot (both in the cross section and > >> also over time for the same manager). I am hoping there must be a way > >> around it using rejection sampling but have not done enough research to > >> comment on how that solution works for such big dimensions. It will be > >> really helpful if you could point me to any specific resources from > >> academia for the same (Haven't been able to find much about random > >> portfolios myself except Portfolio Analytics and Patrick Burns work on > >> Portfolio Probe). As a side note - Do you think translating it using > Rcpp > >> would be time well spent or you think there must be a smarter way to get > >> around it still using R? > >> > >> I really appreciate your help on this thread. > >> > >> Regards, > >> Kshitij Dhingra > >> > >> On Mon, Mar 20, 2017 at 7:21 PM, Brian G. Peterson <[hidden email] > > > >> wrote: > >> > >>> For this type of problem, I would probably generate one set of random > >>> portfolios and just reuse that set of feasible portfolios... My usual > >>> rule > >>> is n-assets + 1-2k feasible portfolios. You can get a better number > e.g. > >>> from sampling theory, but this should be enough. > >>> > >>> Once you have this weights matrix rp, you only need to regenerate rp if > >>> your universe changes. > >>> > >>> Still interested in a more efficient implementation, of course, or we > can > >>> work with you to see if we can find resources to work on it, e.g. from > >>> academia. > >>> > >>> Regards, > >>> > >>> Brian > >>> > >>> > >>> On 03/20/2017 05:28 PM, Kevin Dhingra wrote: > >>> > >>>> Hi Ross, > >>>> > >>>> Sure. Even though I have not profiled the bottlenecks quite in detail > as > >>>> of > >>>> yet, i will give you a decent idea of the problem I am working with. I > >>>> can > >>>> have multiple indices with as much as 2000 assets with group, position > >>>> and > >>>> turnover limits (Not sure if i can increase the speed by removing > >>>> constraints and doing rejection sampling later). In order to generate > a > >>>> daily possible set for the market in this case, I was playing around > >>>> with > >>>> ~4-5 thousand permutations. Also I think I will end up using the > >>>> "sample" > >>>> method because of the type of constraints we have and as you already > >>>> have > >>>> mentioned that method is the slowest (takes about 30 times the time > >>>> using > >>>> "simplex" for the same constraints). Adding box and position limit > >>>> constraints are causing it to run a bit slower (but its not a big > >>>> difference). I can always provide a more thorough analysis of the > >>>> potential > >>>> bottlenecks with a lot more detail when I have a chance to start > working > >>>> on > >>>> translating it to cpp > >>>> > >>>> Thank you, > >>>> > >>>> On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett < > [hidden email]> > >>>> wrote: > >>>> > >>>> Kevin, > >>>>> > >>>>> > >>>>> Can you give us a sense of the number of assets in the portfolio and > >>>>> the constraints? That will help us understand where the potential > >>>>> bottlenecks are in the random portfolio generation. For example, > >>>>> generating a set of random portfolios for box and weight constraints > >>>>> if relatively fast, but adding group or position limit constraints > >>>>> makes the algorithm more complicated and slower. > >>>>> > >>>>> Thanks, > >>>>> Ross > >>>>> > >>>>> > >>>>> On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra > >>>>> <[hidden email]> wrote: > >>>>> > >>>>>> Brian, > >>>>>> > >>>>>> Thank you for a quick reply. I will soon be working on that problem > >>>>>> and > >>>>>> from what I have played with so far, it is unlikely that for our > >>>>>> example > >>>>>> ~2k portfolios will be enough (really hoping it would) to get a good > >>>>>> > >>>>> sense > >>>>> > >>>>>> of the feasible space and seems like I need to implement an Rcpp > >>>>>> version > >>>>>> > >>>>> of > >>>>> > >>>>>> the random portfolios function. I will be happy to collaborate and > >>>>>> share > >>>>>> > >>>>> my > >>>>> > >>>>>> code once i get a decent handle on it locally for the purposes of > our > >>>>>> current project. > >>>>>> > >>>>>> Regards, > >>>>>> Kshitij Dhingra > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson > >>>>>> <[hidden email] > >>>>>>> > >>>>>>> > >>>>>> wrote: > >>>>>> > >>>>>> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: > >>>>>>> > >>>>>>> > >>>>>>>> I have been using the random_portfolios function from the > >>>>>>>> `PortfolioAnalytics` package to simulate the range of > possibilities > >>>>>>>> for return paths at each step under various portfolio constraints > / > >>>>>>>> mandates for evaluating mutual fund managers. As more managers are > >>>>>>>> added to the universe, however, and more simulations are needed, > the > >>>>>>>> pure R implementations get pretty heavy and hard to scale. I was > >>>>>>>> wondering if there has been any work out there thus far on > >>>>>>>> implementing any of the three random portfolio generation methods > >>>>>>>> (sample, simplex, and grid search) at a lower level, using > something > >>>>>>>> like `Rcpp` to enhance the efficiency of these algorithms? > >>>>>>>> > >>>>>>> > >>>>>>> We've discussed it, but I can't say that it is terribly high on our > >>>>>>> list of priorities. > >>>>>>> > >>>>>>> In most cases, no more than 1-2k portfolios should be required to > get > >>>>>>> a > >>>>>>> fair view of the feasible space given your constraints and > >>>>>>> objectives. > >>>>>>> > >>>>>>> We'd be happy to work with you if you want to craft a patch to use > C > >>>>>>> or > >>>>>>> Rcpp for this. > >>>>>>> > >>>>>>> Regards, > >>>>>>> > >>>>>>> Brian > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Kshitij Dhingra > >>>>>> Applied Academics LLC > >>>>>> Office: +1.917.262.0516 > >>>>>> Mobile: +1.206.696.5945 > >>>>>> Email: [hidden email] > >>>>>> Website: http://www.AppliedAcademics.com > >>>>>> > >>>>>> [[alternative HTML version deleted]] > >>>>>> > >>>>>> _______________________________________________ > >>>>>> [hidden email] mailing list > >>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance > >>>>>> -- Subscriber-posting only. If you want to post, subscribe first. > >>>>>> -- Also note that this is not the r-help list where general R > >>>>>> questions > >>>>>> > >>>>> should go. > >>>>> > >>>>> > >>>> > >>>> > >>>> > >>> > >>> -- > >>> Brian G. Peterson > >>> http://braverock.com/brian/ > >>> Ph: 773-459-4973 > >>> IM: bgpbraverock > >>> > >>> > >>> _______________________________________________ > >>> [hidden email] mailing list > >>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance > >>> -- Subscriber-posting only. If you want to post, subscribe first. > >>> -- Also note that this is not the r-help list where general R questions > >>> should go. > >>> > >> > >> > >> > > > > > > -- > > Brian G. Peterson > > http://braverock.com/brian/ > > Ph: 773-459-4973 > > IM: bgpbraverock > > > > _______________________________________________ > > [hidden email] mailing list > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > > -- Subscriber-posting only. If you want to post, subscribe first. > > -- Also note that this is not the r-help list where general R questions > > should go. > > _______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions > should go. > -- Kshitij Dhingra Applied Academics LLC Office: +1.917.262.0516 Mobile: +1.206.696.5945 Email: [hidden email] Website: http://www.AppliedAcademics.com [[alternative HTML version deleted]] _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
In reply to this post by braverock
On Tue, Mar 21, 2017 at 5:41 AM, Brian G. Peterson <[hidden email]> wrote:
> <snip> > To answer the question of whether Rcpp will help is somewhat complex. I'm > confident that some of the nested loops in the generation code will be sped > up by Rcpp. It is possible that more efficient algorithms are available for > constructing the weight vectors. A reason that this hasn't been a huge > priority though is that construction of the random portfolio matrix is > usually not the time limiter in a large optimization: your objective > function is. I think it will be possible to improve the efficiency of this > step, though it is unclear how much of an impact this should have in > practice to a large and complicated numerically solved portfolio > optimization problem. > or not moving to compiled code is worth it is mostly an empirical question. And it's difficult to do more that speculate, unless you have profiling data. So I would strongly encourage you to profile your optimization before you change any code. I would be happy to help review the profiling output. > Regards, > > Brian > > <snip> > > > -- > Brian G. Peterson > http://braverock.com/brian/ > Ph: 773-459-4973 > IM: bgpbraverock > > _______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions > should go. -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com R/Finance 2017 | www.rinfinance.com _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
FYI- I have found it very useful to use random portfolios to verify that
your optimization is actually doing something. Particular mandates get so bogged down by constraints (liquidity, box, sector, industry, Barbra risk, etc) that every answer in the feasible set is basically the same. (i.e. If you are making money it's on the constraints.) - Scott On Tue, Mar 21, 2017 at 13:43 Joshua Ulrich <[hidden email]> wrote: > On Tue, Mar 21, 2017 at 5:41 AM, Brian G. Peterson <[hidden email]> > wrote: > > > <snip> > > To answer the question of whether Rcpp will help is somewhat complex. I'm > > confident that some of the nested loops in the generation code will be > sped > > up by Rcpp. It is possible that more efficient algorithms are available > for > > constructing the weight vectors. A reason that this hasn't been a huge > > priority though is that construction of the random portfolio matrix is > > usually not the time limiter in a large optimization: your objective > > function is. I think it will be possible to improve the efficiency of > this > > step, though it is unclear how much of an impact this should have in > > practice to a large and complicated numerically solved portfolio > > optimization problem. > > > Brian has hinted at this, but I want to say it explicitly. Whether > or not moving to compiled code is worth it is mostly an empirical > question. And it's difficult to do more that speculate, unless you > have profiling data. So I would strongly encourage you to profile > your optimization before you change any code. I would be happy to > help review the profiling output. > > > Regards, > > > > Brian > > > > > <snip> > > > > > > -- > > Brian G. Peterson > > http://braverock.com/brian/ > > Ph: 773-459-4973 > > IM: bgpbraverock > > > > _______________________________________________ > > [hidden email] mailing list > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > > -- Subscriber-posting only. If you want to post, subscribe first. > > -- Also note that this is not the r-help list where general R questions > > should go. > > > > -- > Joshua Ulrich | about.me/joshuaulrich > FOSS Trading | www.fosstrading.com > R/Finance 2017 | www.rinfinance.com > > _______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions > should go. > Scott Payseur [[alternative HTML version deleted]] _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
In reply to this post by braverock
I'm coming significantly late to the thread, but I'll throw in a bit of
my view on the required number of random portfolios to generate. I think it depends a whole lot on what you are doing. If you are essentially looking for a p-value, then you'll need a few hundred at most. You shouldn't really care if the p-value is 4.3% versus 4.1%, and if you're looking at tiny p-values, then algorithmic limitations and the subtleties of what "random" means in the particular situation are going to start to have big impacts. If you are looking at densities, then something like 10,000 portfolios is not terrible. If you are looking at tails, then you want lots more. But again, you have lots of model risk in the tails. Pat On 20/03/2017 23:21, Brian G. Peterson wrote: > For this type of problem, I would probably generate one set of random > portfolios and just reuse that set of feasible portfolios... My usual > rule is n-assets + 1-2k feasible portfolios. You can get a better > number e.g. from sampling theory, but this should be enough. > > Once you have this weights matrix rp, you only need to regenerate rp if > your universe changes. > > Still interested in a more efficient implementation, of course, or we > can work with you to see if we can find resources to work on it, e.g. > from academia. > > Regards, > > Brian > > On 03/20/2017 05:28 PM, Kevin Dhingra wrote: >> Hi Ross, >> >> Sure. Even though I have not profiled the bottlenecks quite in detail >> as of >> yet, i will give you a decent idea of the problem I am working with. I >> can >> have multiple indices with as much as 2000 assets with group, position >> and >> turnover limits (Not sure if i can increase the speed by removing >> constraints and doing rejection sampling later). In order to generate a >> daily possible set for the market in this case, I was playing around with >> ~4-5 thousand permutations. Also I think I will end up using the "sample" >> method because of the type of constraints we have and as you already have >> mentioned that method is the slowest (takes about 30 times the time using >> "simplex" for the same constraints). Adding box and position limit >> constraints are causing it to run a bit slower (but its not a big >> difference). I can always provide a more thorough analysis of the >> potential >> bottlenecks with a lot more detail when I have a chance to start >> working on >> translating it to cpp >> >> Thank you, >> >> On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett <[hidden email]> >> wrote: >> >>> Kevin, >>> >>> Can you give us a sense of the number of assets in the portfolio and >>> the constraints? That will help us understand where the potential >>> bottlenecks are in the random portfolio generation. For example, >>> generating a set of random portfolios for box and weight constraints >>> if relatively fast, but adding group or position limit constraints >>> makes the algorithm more complicated and slower. >>> >>> Thanks, >>> Ross >>> >>> >>> On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra >>> <[hidden email]> wrote: >>>> Brian, >>>> >>>> Thank you for a quick reply. I will soon be working on that problem and >>>> from what I have played with so far, it is unlikely that for our >>>> example >>>> ~2k portfolios will be enough (really hoping it would) to get a good >>> sense >>>> of the feasible space and seems like I need to implement an Rcpp >>>> version >>> of >>>> the random portfolios function. I will be happy to collaborate and >>>> share >>> my >>>> code once i get a decent handle on it locally for the purposes of our >>>> current project. >>>> >>>> Regards, >>>> Kshitij Dhingra >>>> >>>> >>>> >>>> On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson >>>> <[hidden email]> >>>> wrote: >>>> >>>>> On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote: >>>>>> I have been using the random_portfolios function from the >>>>>> `PortfolioAnalytics` package to simulate the range of possibilities >>>>>> for return paths at each step under various portfolio constraints / >>>>>> mandates for evaluating mutual fund managers. As more managers are >>>>>> added to the universe, however, and more simulations are needed, the >>>>>> pure R implementations get pretty heavy and hard to scale. I was >>>>>> wondering if there has been any work out there thus far on >>>>>> implementing any of the three random portfolio generation methods >>>>>> (sample, simplex, and grid search) at a lower level, using something >>>>>> like `Rcpp` to enhance the efficiency of these algorithms? >>>>> >>>>> We've discussed it, but I can't say that it is terribly high on our >>>>> list of priorities. >>>>> >>>>> In most cases, no more than 1-2k portfolios should be required to >>>>> get a >>>>> fair view of the feasible space given your constraints and objectives. >>>>> >>>>> We'd be happy to work with you if you want to craft a patch to use >>>>> C or >>>>> Rcpp for this. >>>>> >>>>> Regards, >>>>> >>>>> Brian >>>>> >>>> >>>> >>>> >>>> -- >>>> Kshitij Dhingra >>>> Applied Academics LLC >>>> Office: +1.917.262.0516 >>>> Mobile: +1.206.696.5945 >>>> Email: [hidden email] >>>> Website: http://www.AppliedAcademics.com >>>> >>>> [[alternative HTML version deleted]] >>>> >>>> _______________________________________________ >>>> [hidden email] mailing list >>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance >>>> -- Subscriber-posting only. If you want to post, subscribe first. >>>> -- Also note that this is not the r-help list where general R questions >>> should go. >>> >> >> >> > > -- Patrick Burns [hidden email] http://www.burns-stat.com http://www.portfolioprobe.com/blog twitter: @burnsstat @portfolioprobe _______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go. |
Free forum by Nabble | Edit this page |