apply.paramset stopping on condition

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

apply.paramset stopping on condition

Diego Peroni
Hello everybody,

I'm using quantstrat apply.paramset to optimize some parameters but my
simulations runs for a long time...

I'm trying to stop bad simulations BEFORE they finish basing on some
indicators (ex. Net.Profit or drawdown).

Is there a way to control this beaviour including some custom code in
quantstrat?

Thanks

Diego

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: apply.paramset stopping on condition

Frank-2
Would you attach an example to your post? Something that we can run using
copy and paste to R console, with some data?

Thanks,

Frank
Chicago

-----Original Message-----
From: R-SIG-Finance [mailto:[hidden email]] On Behalf
Of Diego Peroni
Sent: Tuesday, March 14, 2017 2:51 AM
To: [hidden email]
Subject: [R-SIG-Finance] apply.paramset stopping on condition

Hello everybody,

I'm using quantstrat apply.paramset to optimize some parameters but my
simulations runs for a long time...

I'm trying to stop bad simulations BEFORE they finish basing on some
indicators (ex. Net.Profit or drawdown).

Is there a way to control this beaviour including some custom code in
quantstrat?

Thanks

Diego

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions
should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: apply.paramset stopping on condition

Diego Peroni
Hi Frank,

I don't have code to share: every simulation is fine...

In other words my problem is:

1) In which function of quantstrat can I insert some custom code to stop
simulation execution at a specific moment based on my personal raw
criteria (ex. Profit or Drawdown)?

2) The custom code must be placed in a place where I can collect last
trade result step by step to calculate approximately "partial" Profit or
Drawdown

I'm looking at "ruleOrderProc.R" function but I'm not sure it is the
best place...

Thanks in advance

Diego



On 14/03/2017 17:35, Frank wrote:

> Would you attach an example to your post? Something that we can run using
> copy and paste to R console, with some data?
>
> Thanks,
>
> Frank
> Chicago
>
> -----Original Message-----
> From: R-SIG-Finance [mailto:[hidden email]] On Behalf
> Of Diego Peroni
> Sent: Tuesday, March 14, 2017 2:51 AM
> To: [hidden email]
> Subject: [R-SIG-Finance] apply.paramset stopping on condition
>
> Hello everybody,
>
> I'm using quantstrat apply.paramset to optimize some parameters but my
> simulations runs for a long time...
>
> I'm trying to stop bad simulations BEFORE they finish basing on some
> indicators (ex. Net.Profit or drawdown).
>
> Is there a way to control this beaviour including some custom code in
> quantstrat?
>
> Thanks
>
> Diego
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: apply.paramset stopping on condition

braverock
How are you planning to calculate this statistic to 'stop trading'?

If you're using net profit, drawdown, etc., these are typically
calculated after the backtest is over, e.g. using updatePortf and
tradeStats.

Obviously this is too late to bail out early.

You could conceivably check some 'partial' profit or loss in a rule
(maybe a rebalance rule) from the trades that have happened, zero out
all your signals, adjust the next dindex to the last observation to
skip to the end.

I'd argue that your question suggests that you're not doing analysis of
 your signals before running the expensive walk-forward or
apply.paramset processes.  Wouldn't it make more sense to apply your
proposed parameters over your signal process, where the code should be
vectorized, fast, and cheap, than to wait till you're running the
comparatively slower path dependent rules?

quantstrat has many functions for doing signal analysis, including
signal analysis over a parameter set.  See the signal analysis demos.

Also, I suspect you may be doing something else computationally
expensive in your backtest. We can typically get to about one core-
minute per symbol per day on tick data.  So we've certainly seen tests
that take hours or even days to run here, but those are over massive
amounts of data.  Have you profiled a single run of the backtest to see
where it is spending its time?

Regards,

Brian


--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

On Tue, 2017-03-14 at 17:52 +0100, Diego Peroni wrote:

> Hi Frank,
>
> I don't have code to share: every simulation is fine...
>
> In other words my problem is:
>
> 1) In which function of quantstrat can I insert some custom code to
> stop 
> simulation execution at a specific moment based on my personal raw 
> criteria (ex. Profit or Drawdown)?
>
> 2) The custom code must be placed in a place where I can collect
> last 
> trade result step by step to calculate approximately "partial" Profit
> or 
> Drawdown
>
> I'm looking at "ruleOrderProc.R" function but I'm not sure it is the 
> best place...
>
> Thanks in advance
>
> Diego
>
>
>
> On 14/03/2017 17:35, Frank wrote:
> > Would you attach an example to your post? Something that we can run
> > using
> > copy and paste to R console, with some data?
> >
> > Thanks,
> >
> > Frank
> > Chicago
> >
> > -----Original Message-----
> > From: R-SIG-Finance [mailto:[hidden email]] On
> > Behalf
> > Of Diego Peroni
> > Sent: Tuesday, March 14, 2017 2:51 AM
> > To: [hidden email]
> > Subject: [R-SIG-Finance] apply.paramset stopping on condition
> >
> > Hello everybody,
> >
> > I'm using quantstrat apply.paramset to optimize some parameters but
> > my
> > simulations runs for a long time...
> >
> > I'm trying to stop bad simulations BEFORE they finish basing on
> > some
> > indicators (ex. Net.Profit or drawdown).
> >
> > Is there a way to control this beaviour including some custom code
> > in
> > quantstrat?
> >
> > Thanks
> >
> > Diego
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R
> > questions
> > should go.
> >
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R
> questions should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: apply.paramset stopping on condition

Diego Peroni
Hi Brain,

thanks for your answers.

See following answers below.

Diego


On 14/03/2017 18:02, Brian G. Peterson wrote:
> How are you planning to calculate this statistic to 'stop trading'?
For me it's just a threshold for drawdown: reading transaction by
transaction I can easly calculate it.

>
> If you're using net profit, drawdown, etc., these are typically
> calculated after the backtest is over, e.g. using updatePortf and
> tradeStats.
>
> Obviously this is too late to bail out early.
>
> You could conceivably check some 'partial' profit or loss in a rule
> (maybe a rebalance rule) from the trades that have happened, zero out
> all your signals, adjust the next dindex to the last observation to
> skip to the end.
Ok this is probably what I'm looking for!
>
> I'd argue that your question suggests that you're not doing analysis of
>   your signals before running the expensive walk-forward or
> apply.paramset processes.  Wouldn't it make more sense to apply your
> proposed parameters over your signal process, where the code should be
> vectorized, fast, and cheap, than to wait till you're running the
> comparatively slower path dependent rules?
If I've understood your suggestion I've seen that indicator&signal
calculation takes just few seconds, it's not a problem.
But I don't know effectiveness over trading...
>
> quantstrat has many functions for doing signal analysis, including
> signal analysis over a parameter set.  See the signal analysis demos.
I'm going to see the code in signal.SMA.R and signal.RSI.R
>
> Also, I suspect you may be doing something else computationally
> expensive in your backtest. We can typically get to about one core-
> minute per symbol per day on tick data.  So we've certainly seen tests
> that take hours or even days to run here, but those are over massive
> amounts of data.  Have you profiled a single run of the backtest to see
> where it is spending its time?
I'm testing 7 years of minute data (1 symbol) and 1 backtesting normally
takes about 2 hours.
My simulation uses trailingstops and I think that it is the slowest
order type (path.dep).
Am I wrong?

Is there a way to profile where my backtest is spending its time?

Thanks a lot for your help! :-)

>
> Regards,
>
> Brian
>
>

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.