R › Rmetrics

Are there genetic algorithm for trading strategy evolution in R?

Classic

List

Threaded

18 messages Options

LosemindL

Are there genetic algorithm for trading strategy evolution in R?

Hi all, Good morning, good afternoon and good evening!

Could anybody please kindly point me to resources in R which shows about
how to use Genetic algorithm to evolve trading strategies?

I did a lot search on Google these days and certainly it's a well-covered
and popular topic, but I don't see anywhere in R...

Thanks a lot!

[[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

Sofian Hadiwijaya

Re: Are there genetic algorithm for trading strategy evolution in R?

how about quantmod library..

On Wed, Mar 7, 2012 at 10:30 PM, Michael <[hidden email]> wrote:

> Hi all, Good morning, good afternoon and good evening!
>
> Could anybody please kindly point me to resources in R which shows about
> how to use Genetic algorithm to evolve trading strategies?
>
> I did a lot search on Google these days and certainly it's a well-covered
> and popular topic, but I don't see anywhere in R...
>
> Thanks a lot!
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

[[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

Zachary Mayer

Re: Are there genetic algorithm for trading strategy evolution in R?

There is the DEoptim<http://cran.r-project.org/web/packages/DEoptim/index.html>library
in r, which is an excellent library for differential evolution. If
you can define your trading strategy in terms of a bunch of parameters to
adjust and an objective function (i.e. turn it into an optimization
problem), DEoptim will help you find the minimum (or maximum).

DEoptim works well on non-differentiable problems with many local minima.
Here is an example of using it to solve a portfolio optimization problem:
http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf

On Thu, Mar 8, 2012 at 12:43 AM, Sofian Hadiwijaya <[hidden email]>wrote:

> how about quantmod library..
>
> On Wed, Mar 7, 2012 at 10:30 PM, Michael <[hidden email]> wrote:
>
> > Hi all, Good morning, good afternoon and good evening!
> >
> > Could anybody please kindly point me to resources in R which shows about
> > how to use Genetic algorithm to evolve trading strategies?
> >
> > I did a lot search on Google these days and certainly it's a well-covered
> > and popular topic, but I don't see anywhere in R...
> >
> > Thanks a lot!
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions
> > should go.
> >
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

braverock

Re: Are there genetic algorithm for trading strategy evolution in R?

On Thu, 2012-03-08 at 09:25 -0500, Zachary Mayer wrote:
> There is the DEoptim<http://cran.r-project.org/web/packages/DEoptim/index.html>library
> in r, which is an excellent library for differential evolution. If
> you can define your trading strategy in terms of a bunch of parameters to
> adjust and an objective function (i.e. turn it into an optimization
> problem), DEoptim will help you find the minimum (or maximum).
>
> DEoptim works well on non-differentiable problems with many local minima.
> Here is an example of using it to solve a portfolio optimization problem:
> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf

Certainly DEoptim can be used for parameter optimization of a trading
strategy. Of course focusing on genetic algorithms to the exclusion of
other global optimization algorithms may miss the larger challenges of
optimizing trading system parameters.

I think one of the biggest challenges in this space is the definition of
an objective function to give to the optimizer. What constitutes
success? (and I'm sure your answer will vary widely from other answers
to this question, not one size fits all).

Another major challenge here is that parameter optimization gets into
two 'hard' optimization problems: mixed integer problems, and
multi-objective optimization.

Most global optimizers use some random/directed selection in continuous
floating point numbers. Trading system parameters may be integers,
factors, floating point, etc. This poses some difficulty in fitting the
optimizer's floating point parameters to the trading system.

Also, you may actually have multiple objectives. Maximize Returns and
minimize risk, subject to some maximum drawdown. This means that you
need to optimize for all these objectives simultaneously. In portfolio
problems, this is reasonably well understood, see for example
PortfolioAnalytics on R-Forge. In trading system optimization, it is
not nearly so clear how to create an appropriate penalized objective for
the optimizer to work on.

Regards,

- Brian

--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

LosemindL

Re: Are there genetic algorithm for trading strategy evolution in R?

In reply to this post by Zachary Mayer

Thanks folks!

After digging further on the Internet, I have the following questions:

Q1: I read the following article:

http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf

It seems that there are a bunch of parameters in this optimizer and the
results are sensitive to these parameters.

So there is another layer of optimization with respect to these optimizer
parameters.

Is the "tweaking" of these optimizer parameters data-mining, which will
lead to data-snooping bias?

Q2: Due to the random nature of the optimizer, each time you run the
backtest, you will have different performance.

What do you do in that case?

So for out-of-sample real-trading, we are trading a random strategy?

Q3: It's pretty easy to understand using Genetic Algorithms to serve as a
replacement for regular optimizers;

but using Genetic Algorithms to evolve trading strategies seem to be
different. Anywhere we could find such an example in R?

On Thu, Mar 8, 2012 at 8:25 AM, Zachary Mayer <[hidden email]> wrote:

> There is the DEoptim<http://cran.r-project.org/web/packages/DEoptim/index.html>library in r, which is an excellent library for differential evolution. If
> you can define your trading strategy in terms of a bunch of parameters to
> adjust and an objective function (i.e. turn it into an optimization
> problem), DEoptim will help you find the minimum (or maximum).
>
> DEoptim works well on non-differentiable problems with many local minima.
> Here is an example of using it to solve a portfolio optimization problem:
>
> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
>
>
>
> On Thu, Mar 8, 2012 at 12:43 AM, Sofian Hadiwijaya <[hidden email]>wrote:
>
>> how about quantmod library..
>>
>> On Wed, Mar 7, 2012 at 10:30 PM, Michael <[hidden email]> wrote:
>>
>> > Hi all, Good morning, good afternoon and good evening!
>> >
>> > Could anybody please kindly point me to resources in R which shows about
>> > how to use Genetic algorithm to evolve trading strategies?
>> >
>> > I did a lot search on Google these days and certainly it's a
>> well-covered
>> > and popular topic, but I don't see anywhere in R...
>> >
>> > Thanks a lot!
>> >
>> > [[alternative HTML version deleted]]
>> >
>> > _______________________________________________
>> > [hidden email] mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> > -- Subscriber-posting only. If you want to post, subscribe first.
>> > -- Also note that this is not the r-help list where general R questions
>> > should go.
>> >
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions
>> should go.
>>
>
>

Luna Moon

Re: Are there genetic algorithm for trading strategy evolution in R?

In reply to this post by braverock

Thanks Brian. Those are very good points! I concur with Brian.

Hence my Q4:

What's a good objective function for using with any optimizer?

Intuitively there should be at least two objectives:

1. In the past (in-sample data), the performance should be the best (be it
Sharpe, or MDD, etc.)

2. In the future (out-sample data), the performance should still be
good(hopefully), i.e. there should be some sort of "staying power"...

Therefore, if we divide the whole data into three parts:

1. In-sample

2. Validation

3. Out-sample

Both part 1 and part 2 data are known at the time of optimization.

Then we can use part 1 and part 2 data to optimize for both "performance"
and "staying-power".

My questions are:

1. Does this procedure make sense?

2. What is a good measure for "staying power" on the "Validation" part of
data?

3. With two measures "performance" and "staying power", how do we weigh
them and transform them into one measure and therefore one objective
function.

Any thoughts?

Thanks a lot!

On Thu, Mar 8, 2012 at 8:41 AM, Brian G. Peterson <[hidden email]>wrote:

>
> On Thu, 2012-03-08 at 09:25 -0500, Zachary Mayer wrote:
> > There is the DEoptim<
> http://cran.r-project.org/web/packages/DEoptim/index.html>library
> > in r, which is an excellent library for differential evolution. If
> > you can define your trading strategy in terms of a bunch of parameters to
> > adjust and an objective function (i.e. turn it into an optimization
> > problem), DEoptim will help you find the minimum (or maximum).
> >
> > DEoptim works well on non-differentiable problems with many local minima.
> > Here is an example of using it to solve a portfolio optimization
> problem:
> >
> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
>
> Certainly DEoptim can be used for parameter optimization of a trading
> strategy. Of course focusing on genetic algorithms to the exclusion of
> other global optimization algorithms may miss the larger challenges of
> optimizing trading system parameters.
>
> I think one of the biggest challenges in this space is the definition of
> an objective function to give to the optimizer. What constitutes
> success? (and I'm sure your answer will vary widely from other answers
> to this question, not one size fits all).
>
> Another major challenge here is that parameter optimization gets into
> two 'hard' optimization problems: mixed integer problems, and
> multi-objective optimization.
>
> Most global optimizers use some random/directed selection in continuous
> floating point numbers. Trading system parameters may be integers,
> factors, floating point, etc. This poses some difficulty in fitting the
> optimizer's floating point parameters to the trading system.
>
> Also, you may actually have multiple objectives. Maximize Returns and
> minimize risk, subject to some maximum drawdown. This means that you
> need to optimize for all these objectives simultaneously. In portfolio
> problems, this is reasonably well understood, see for example
> PortfolioAnalytics on R-Forge. In trading system optimization, it is
> not nearly so clear how to create an appropriate penalized objective for
> the optimizer to work on.
>
> Regards,
>
> - Brian
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

Worik Stanton

Re: Are there genetic algorithm for trading strategy evolution in R?

In reply to this post by LosemindL

On 09/03/12 07:16, Michael wrote:

> Thanks folks!
>
> After digging further on the Internet, I have the following questions:
>
> Q1: I read the following article:
>
> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
>
> It seems that there are a bunch of parameters in this optimizer and the
> results are sensitive to these parameters.
>
> So there is another layer of optimization with respect to these optimizer
> parameters.
>
> Is the "tweaking" of these optimizer parameters data-mining, which will
> lead to data-snooping bias?

Yes. But that is the same as any sort of parameter search.

I had a *brief* look at that paper just now and they seem to be using
monthly returns. So they will have ~240 data points over 20 years.
Ignoring the fact that optimisation 15 years ago may well not be the
same as now (markets may change) that is still not much data once you
embark on parameter optimisation.

To avoid data snooping there is really no replacement for more data, (or
avoid search!)
> Q2: Due to the random nature of the optimizer, each time you run the
> backtest, you will have different performance.
>
> What do you do in that case?
?set.seed

cheers
Worik

> So for out-of-sample real-trading, we are trading a random strategy?
>
> Q3: It's pretty easy to understand using Genetic Algorithms to serve as a
> replacement for regular optimizers;
>
> but using Genetic Algorithms to evolve trading strategies seem to be
> different. Anywhere we could find such an example in R?
>
>
>
>
> On Thu, Mar 8, 2012 at 8:25 AM, Zachary Mayer<[hidden email]> wrote:
>
>> There is the DEoptim<http://cran.r-project.org/web/packages/DEoptim/index.html>library in r, which is an excellent library for differential evolution. If
>> you can define your trading strategy in terms of a bunch of parameters to
>> adjust and an objective function (i.e. turn it into an optimization
>> problem), DEoptim will help you find the minimum (or maximum).
>>
>> DEoptim works well on non-differentiable problems with many local minima.
>> Here is an example of using it to solve a portfolio optimization problem:
>>
>> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
>>
>>
>>
>> On Thu, Mar 8, 2012 at 12:43 AM, Sofian Hadiwijaya<[hidden email]>wrote:
>>
>>> how about quantmod library..
>>>
>>> On Wed, Mar 7, 2012 at 10:30 PM, Michael<[hidden email]> wrote:
>>>
>>>> Hi all, Good morning, good afternoon and good evening!
>>>>
>>>> Could anybody please kindly point me to resources in R which shows about
>>>> how to use Genetic algorithm to evolve trading strategies?
>>>>
>>>> I did a lot search on Google these days and certainly it's a
>>> well-covered
>>>> and popular topic, but I don't see anywhere in R...
>>>>
>>>> Thanks a lot!
>>>>
>>>> [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>> -- Also note that this is not the r-help list where general R questions
>>>> should go.
>>>>
>>> [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R questions
>>> should go.
>>>
>>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>

--
Foo!

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

Patrick Burns-2

Re: Are there genetic algorithm for trading strategy evolution in R?

In reply to this post by LosemindL

Comments inline.

On 08/03/2012 18:16, Michael wrote:

I wouldn't think so, but there might be
a way to manage it.

>
> Q2: Due to the random nature of the optimizer, each time you run the
> backtest, you will have different performance.
>
> What do you do in that case?

That may be a good thing, if you are
willing to use it.

In what I've done on backtesting:

http://www.portfolioprobe.com/2010/11/05/backtesting-almost-wordless/

I show how to assess whether the strategy
is better than luck by using random trades.

The standard thing to assume (as I do in
that piece) is that the optimization is
noiseless. But really the optimization
depends on a multitude of subtle influences.
Even if you always got the exact global
optimum, if a variance or expected return
were slightly different, you could get a
very different path. The "optimal" path
is fuzzy in actuality.

>
> So for out-of-sample real-trading, we are trading a random strategy?

Yes. But the inputs are random so even
non-stochastic optimizers give you a
random strategy in a sense.

>
> Q3: It's pretty easy to understand using Genetic Algorithms to serve as a
> replacement for regular optimizers;
>
> but using Genetic Algorithms to evolve trading strategies seem to be
> different. Anywhere we could find such an example in R?

Yes, that is different.

In https://stat.ethz.ch/pipermail/r-sig-finance/2010q4/007033.html
you can find Josh quoting me quoting Lao-Tzu
on why you are unlikely to find much useful
on that subject.

Pat

>
>
>
>
> On Thu, Mar 8, 2012 at 8:25 AM, Zachary Mayer<[hidden email]> wrote:
>
>> There is the DEoptim<http://cran.r-project.org/web/packages/DEoptim/index.html>library in r, which is an excellent library for differential evolution. If
>> you can define your trading strategy in terms of a bunch of parameters to
>> adjust and an objective function (i.e. turn it into an optimization
>> problem), DEoptim will help you find the minimum (or maximum).
>>
>> DEoptim works well on non-differentiable problems with many local minima.
>> Here is an example of using it to solve a portfolio optimization problem:
>>
>> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
>>
>>
>>
>> On Thu, Mar 8, 2012 at 12:43 AM, Sofian Hadiwijaya<[hidden email]>wrote:
>>
>>> how about quantmod library..
>>>
>>> On Wed, Mar 7, 2012 at 10:30 PM, Michael<[hidden email]> wrote:
>>>
>>>> Hi all, Good morning, good afternoon and good evening!
>>>>
>>>> Could anybody please kindly point me to resources in R which shows about
>>>> how to use Genetic algorithm to evolve trading strategies?
>>>>
>>>> I did a lot search on Google these days and certainly it's a
>>> well-covered
>>>> and popular topic, but I don't see anywhere in R...
>>>>
>>>> Thanks a lot!
>>>>
>>>> [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>> -- Also note that this is not the r-help list where general R questions
>>>> should go.
>>>>
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R questions
>>> should go.
>>>
>>
>>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>

Daniel Cegiełka

Re: Are there genetic algorithm for trading strategy evolution in R?

Let me add my two cents. Old Max Dama blog (mirror):

http://smartdatacollective.com/maxdama/22571/voodoo-spectrum-machine-learning-and-data-sets

Optimization is good to examine the sensitivity of the model and the
selection of appropriate parameters - this is useful. But playing with the
evolutionary strategy, what you Michael ask, is very risky.

regards,
Daniel

2012/3/8 Patrick Burns <[hidden email]>

> Comments inline.
>
>
> On 08/03/2012 18:16, Michael wrote:
>
>> Thanks folks!
>>
>> After digging further on the Internet, I have the following questions:
>>
>> Q1: I read the following article:
>>
>> http://cran.r-project.org/web/**packages/DEoptim/vignettes/**
>> DEoptimPortfolioOptimization.**pdf<http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf>
>>
>> It seems that there are a bunch of parameters in this optimizer and the
>> results are sensitive to these parameters.
>>
>> So there is another layer of optimization with respect to these optimizer
>> parameters.
>>
>> Is the "tweaking" of these optimizer parameters data-mining, which will
>> lead to data-snooping bias?
>>
>
> I wouldn't think so, but there might be
> a way to manage it.
>
>
>
>> Q2: Due to the random nature of the optimizer, each time you run the
>> backtest, you will have different performance.
>>
>> What do you do in that case?
>>
>
> That may be a good thing, if you are
> willing to use it.
>
> In what I've done on backtesting:
>
> http://www.portfolioprobe.com/**2010/11/05/backtesting-almost-**wordless/<http://www.portfolioprobe.com/2010/11/05/backtesting-almost-wordless/>
>
> I show how to assess whether the strategy
> is better than luck by using random trades.
>
> The standard thing to assume (as I do in
> that piece) is that the optimization is
> noiseless. But really the optimization
> depends on a multitude of subtle influences.
> Even if you always got the exact global
> optimum, if a variance or expected return
> were slightly different, you could get a
> very different path. The "optimal" path
> is fuzzy in actuality.
>
>
>
>> So for out-of-sample real-trading, we are trading a random strategy?
>>
>
> Yes. But the inputs are random so even
> non-stochastic optimizers give you a
> random strategy in a sense.
>
>
>
>> Q3: It's pretty easy to understand using Genetic Algorithms to serve as a
>> replacement for regular optimizers;
>>
>> but using Genetic Algorithms to evolve trading strategies seem to be
>> different. Anywhere we could find such an example in R?
>>
>
> Yes, that is different.
>
> In https://stat.ethz.ch/**pipermail/r-sig-finance/**2010q4/007033.html<https://stat.ethz.ch/pipermail/r-sig-finance/2010q4/007033.html>
> you can find Josh quoting me quoting Lao-Tzu
> on why you are unlikely to find much useful
> on that subject.
>
> Pat
>
>
>
>>
>>
>>
>> On Thu, Mar 8, 2012 at 8:25 AM, Zachary Mayer<[hidden email]>
>> wrote:
>>
>> There is the DEoptim<http://cran.r-project.**org/web/packages/DEoptim/**
>>> index.html <http://cran.r-project.org/web/packages/DEoptim/index.html>>library
>>> in r, which is an excellent library for differential evolution. If
>>> you can define your trading strategy in terms of a bunch of parameters to
>>> adjust and an objective function (i.e. turn it into an optimization
>>> problem), DEoptim will help you find the minimum (or maximum).
>>>
>>> DEoptim works well on non-differentiable problems with many local minima.
>>> Here is an example of using it to solve a portfolio optimization
>>> problem:
>>>
>>> http://cran.r-project.org/web/**packages/DEoptim/vignettes/**
>>> DEoptimPortfolioOptimization.**pdf<http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf>
>>>
>>>
>>>
>>> On Thu, Mar 8, 2012 at 12:43 AM, Sofian Hadiwijaya<reztinpeace@gmail.**
>>> com <[hidden email]>>wrote:
>>>
>>> how about quantmod library..
>>>>
>>>> On Wed, Mar 7, 2012 at 10:30 PM, Michael<[hidden email]> wrote:
>>>>
>>>> Hi all, Good morning, good afternoon and good evening!
>>>>>
>>>>> Could anybody please kindly point me to resources in R which shows
>>>>> about
>>>>> how to use Genetic algorithm to evolve trading strategies?
>>>>>
>>>>> I did a lot search on Google these days and certainly it's a
>>>>>
>>>> well-covered
>>>>
>>>>> and popular topic, but I don't see anywhere in R...
>>>>>
>>>>> Thanks a lot!
>>>>>
>>>>> [[alternative HTML version deleted]]
>>>>>
>>>>> ______________________________**_________________
>>>>> [hidden email] mailing list
>>>>> https://stat.ethz.ch/mailman/**listinfo/r-sig-finance<https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
>>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>>> -- Also note that this is not the r-help list where general R questions
>>>>> should go.
>>>>>
>>>>>
>>>> [[alternative HTML version deleted]]
>>>>
>>>> ______________________________**_________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/**listinfo/r-sig-finance<https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>> -- Also note that this is not the r-help list where general R questions
>>>> should go.
>>>>
>>>>
>>>
>>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________**_________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/**listinfo/r-sig-finance<https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions
>> should go.
>>
>>
> --
> Patrick Burns
> [hidden email]
> http://www.burns-stat.com
> http://www.portfolioprobe.com/**blog <http://www.portfolioprobe.com/blog>
> twitter: @portfolioprobe
>
>
> ______________________________**_________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/**listinfo/r-sig-finance<https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

Patrick Burns-2

Re: Are there genetic algorithm for trading strategy evolution in R?

Yes, serious chance of doing it poorly
without years of intense work.

On 08/03/2012 19:14, Daniel Cegiełka wrote:

> Let me add my two cents. Old Max Dama blog (mirror):
>
> http://smartdatacollective.com/maxdama/22571/voodoo-spectrum-machine-learning-and-data-sets
>
>
> Optimization is good to examine the sensitivity of the model and the
> selection of appropriate parameters - this is useful. But playing with
> the evolutionary strategy, what you Michael ask, is very risky.
>
> regards,
> Daniel
>
>
>
> 2012/3/8 Patrick Burns <[hidden email]
> <mailto:[hidden email]>>
>
> Comments inline.
>
>
> On 08/03/2012 18:16, Michael wrote:
>
> Thanks folks!
>
> After digging further on the Internet, I have the following
> questions:
>
> Q1: I read the following article:
>
> http://cran.r-project.org/web/__packages/DEoptim/vignettes/__DEoptimPortfolioOptimization.__pdf
> <http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf>
>
> It seems that there are a bunch of parameters in this optimizer
> and the
> results are sensitive to these parameters.
>
> So there is another layer of optimization with respect to these
> optimizer
> parameters.
>
> Is the "tweaking" of these optimizer parameters data-mining,
> which will
> lead to data-snooping bias?
>
>
> I wouldn't think so, but there might be
> a way to manage it.
>
>
>
> Q2: Due to the random nature of the optimizer, each time you run the
> backtest, you will have different performance.
>
> What do you do in that case?
>
>
> That may be a good thing, if you are
> willing to use it.
>
> In what I've done on backtesting:
>
> http://www.portfolioprobe.com/__2010/11/05/backtesting-almost-__wordless/
> <http://www.portfolioprobe.com/2010/11/05/backtesting-almost-wordless/>
>
> I show how to assess whether the strategy
> is better than luck by using random trades.
>
> The standard thing to assume (as I do in
> that piece) is that the optimization is
> noiseless. But really the optimization
> depends on a multitude of subtle influences.
> Even if you always got the exact global
> optimum, if a variance or expected return
> were slightly different, you could get a
> very different path. The "optimal" path
> is fuzzy in actuality.
>
>
>
> So for out-of-sample real-trading, we are trading a random strategy?
>
>
> Yes. But the inputs are random so even
> non-stochastic optimizers give you a
> random strategy in a sense.
>
>
>
> Q3: It's pretty easy to understand using Genetic Algorithms to
> serve as a
> replacement for regular optimizers;
>
> but using Genetic Algorithms to evolve trading strategies seem to be
> different. Anywhere we could find such an example in R?
>
>
> Yes, that is different.
>
> In
> https://stat.ethz.ch/__pipermail/r-sig-finance/__2010q4/007033.html
> <https://stat.ethz.ch/pipermail/r-sig-finance/2010q4/007033.html>
> you can find Josh quoting me quoting Lao-Tzu
> on why you are unlikely to find much useful
> on that subject.
>
> Pat
>
>
>
>
>
>
> On Thu, Mar 8, 2012 at 8:25 AM, Zachary
> Mayer<[hidden email] <mailto:[hidden email]>> wrote:
>
> There is the
> DEoptim<http://cran.r-project.__org/web/packages/DEoptim/__index.html
> <http://cran.r-project.org/web/packages/DEoptim/index.html>>library
> in r, which is an excellent library for differential
> evolution. If
> you can define your trading strategy in terms of a bunch of
> parameters to
> adjust and an objective function (i.e. turn it into an
> optimization
> problem), DEoptim will help you find the minimum (or maximum).
>
> DEoptim works well on non-differentiable problems with many
> local minima.
> Here is an example of using it to solve a portfolio
> optimization problem:
>
> http://cran.r-project.org/web/__packages/DEoptim/vignettes/__DEoptimPortfolioOptimization.__pdf
> <http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf>
>
>
>
> On Thu, Mar 8, 2012 at 12:43 AM, Sofian
> Hadiwijaya<reztinpeace@gmail.__com
> <mailto:[hidden email]>>wrote:
>
> how about quantmod library..
>
> On Wed, Mar 7, 2012 at 10:30 PM,
> Michael<[hidden email]
> <mailto:[hidden email]>> wrote:
>
> Hi all, Good morning, good afternoon and good evening!
>
> Could anybody please kindly point me to resources in
> R which shows about
> how to use Genetic algorithm to evolve trading
> strategies?
>
> I did a lot search on Google these days and
> certainly it's a
>
> well-covered
>
> and popular topic, but I don't see anywhere in R...
>
> Thanks a lot!
>
> [[alternative HTML version deleted]]
>
> _________________________________________________
> [hidden email]
> <mailto:[hidden email]> mailing list
> https://stat.ethz.ch/mailman/__listinfo/r-sig-finance <https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> -- Subscriber-posting only. If you want to post,
> subscribe first.
> -- Also note that this is not the r-help list where
> general R questions
> should go.
>
>
> [[alternative HTML version deleted]]
>
> _________________________________________________
> [hidden email]
> <mailto:[hidden email]> mailing list
> https://stat.ethz.ch/mailman/__listinfo/r-sig-finance
> <https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> -- Subscriber-posting only. If you want to post,
> subscribe first.
> -- Also note that this is not the r-help list where
> general R questions
> should go.
>
>
>
>
> [[alternative HTML version deleted]]
>
> _________________________________________________
> [hidden email] <mailto:[hidden email]>
> mailing list
> https://stat.ethz.ch/mailman/__listinfo/r-sig-finance
> <https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R
> questions should go.
>
>
> --
> Patrick Burns
> [hidden email] <mailto:[hidden email]>
> http://www.burns-stat.com
> http://www.portfolioprobe.com/__blog
> <http://www.portfolioprobe.com/blog>
> twitter: @portfolioprobe
>
>
> _________________________________________________
> [hidden email] <mailto:[hidden email]>
> mailing list
> https://stat.ethz.ch/mailman/__listinfo/r-sig-finance
> <https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R
> questions should go.
>
>

Gaurav Malhotra

Re: testing momentum - Quantstrat

In reply to this post by Sofian Hadiwijaya

Hi all,
I'm currently working on a project that tests whether momentum ( as defined by Jegadeesh and Titman, 1993) exists in India.

I'm new to R and ran into problems. I'm hoping someone here can help me with an example snippet of the code where I'm stuck... Thanks in advance!

The trading idea is to rank stocks into deciles based on the past (3,6,12 month) returns, go long the highest decile (thus a portfolio) and short the lowest decile (another portfolio) - standard momentum portfolio from any investments course. The portfolios thus formed are held for 3,6,12 months.

Note that the indicators use returns instead of price. I'm unable to get the ranking done correctly. I have attempted to write a "rank" function in order to rank the securities in any given month/date based on their past 3 (as well as 6 & 7) month returns, but so far have been unable to.

# Load required libraries
library(quantmod)
library(quantstrat)

# Try to clean up in case the demo was run previously
suppressWarnings(rm("account.faber","portfolio.faber",pos=.blotter))
suppressWarnings(rm("ltaccount", "ltportfolio", "ClosePrice", "CurrentDate", "equity",
"GSPC", "stratFaber", "initDate", "initEq", "Posn", "UnitSize", "verbose"))
suppressWarnings(rm("order_book.faber",pos=.strategy))

# Set initial values
initDate='2000-01-01'
initEq=100000

currency("INR")

# Set up instruments with FinancialInstruments package
currency("INR")
symbols= c("AXISBANK.BO","BAJAJAUT.BO","BPCL.BO","BHARTIARTL.BO","CAIRN.BO","CIPLA.BO","COALINDIA.BO","DLF.BO","DRREDDY.BO")

for(symbolinsymbols){ # establish tradable instruments
stock(symbol, currency="INR",multiplier=1)
}

# Load data with quantmod, make monthly
getSymbols(symbols, src='yahoo', index.class=c("POSIXt","POSIXct"), from=initDate)
for(symbolinsymbols) {
x<-get(symbol)
x<-to.monthly(x,indexAt='lastof',drop.time=TRUE)
indexFormat(x)<-'%Y-%m-%d'
colnames(x)<-gsub("x",symbol,colnames(x))
assign(symbol,x)
}

# Initialize portfolio and account
port<-'moment'#use a string here for easier changing of parameters and re-trying
initPortf(port, symbols=symbols, initDate=initDate)
initAcct(port, portfolios=port, initDate=initDate, initEq=100000)
initOrders(portfolio=port, initDate=initDate)

print("setup completed")

### Here is the problem

rank<-function()

# and based on how the rank function turns out, the signals/rules must react accordingly

From what I gathered on the internet, one way to rank was as follows:

getSymbols(c("GE","XOM"), from = "2012-02-20")
x<-cbind(GE,XOM)

NewVar<-cut(x,quantile(x,(0:10)/10),include.lowest=TRUE)

I know this is not working for several reasons, one being that it is ranking by column, but I have little idea how to either correct it or to use any other technique.

How do I modify this so that it goes across columns and ranks them, for a given row.

Thanks!

Gaurav M

[[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

Ulrich Staudinger-4

Re: Are there genetic algorithm for trading strategy evolution in R?

In reply to this post by Patrick Burns-2

Hey guys,
just some netiquette rant from my side ...
If you reply to an email, I would prefer that people cut the original post
to a necessary minimum instead of including all these other lines in it. I
find it extremely annoying when my gmail page is filled with quoted lines
and here and there two lines of actual answer.
Cheers

On Thu, Mar 8, 2012 at 8:24 PM, Patrick Burns <[hidden email]>wrote:

> Yes, serious chance of doing it poorly
> without years of intense work.
>
--
Ulrich Staudinger

<http://goog_958005736>http://www.activequant.com
Connect online: https://www.xing.com/profile/Ulrich_Staudinger

[[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

darko

Re: Are there genetic algorithm for trading strategy evolution in R?

In reply to this post by Patrick Burns-2

To contribute to this discussion with a more concrete example of genetic algorithm usage for trading purpose I have disclosed simple code snippet how to implement it using DEoptim. GALGO is alternative to DEoptim in R.

Micheal, generally I do not provide complete examples so for this exception thank to Patrick Burns as he helped me in the past with PortfolioProbe Optimizer. Hope it helps.

******Disclaimer this is just example for learning purposes but no warranty is made as to accuracy and no liability is accepted if used for commercial purposes****

require(quantmod)
require(PerformanceAnalytics)
require(blotter)
require(DEoptim)
################ MACD TEST
z=read.csv(file="G:\\QERD\\GALGO\\first.csv",header=TRUE,sep=",",stringsAsFactors=FALSE, colClasses=c('character','numeric','numeric'))
z =as.xts(z, as.POSIXct(z$Date))
#force columns into numeric values
z$sum=as.numeric(z$INTC)+as.numeric(z$IEF)
z = z[,c("INTC", "IEF")]
# Let's think about returns instead of prices...
# Ra is the log return for a buy-and-hold strategy, Rb the 'benchmark.'
# We will extensively use these log-return series in the sequel.

z$Ra = Return.calculate(z$INTC)
z$Rb = Return.calculate(z$IEF)
# we will optimise over in sample data
insample=z['2008:']

#the fitness function must obtain the data
MACDFitness <- function(params) {
fast <- params[1]
slow <- params[2]
sig <- params[3]
if ((fast <= slow - 1) & (slow >= 4) & (fast >=4) & ((sig >= fast) & (sig <= slow))) { #certain conditions does not make sense
params = paste("F: ",fast," slow: ",slow,sep="")
x <- MACD(parent$INTC, nFast=fast, nSlow=slow, nSig=sig,maType="EMA")
position <- sign(x[,1]-x[,2])
s <- xts(position,order.by=index(parent))
s$Ra <- parent$Ra
s$Rb <- parent$Rb
s$rts <- (s$Ra*(s$macd>0)) + (s$Rb*(s$macd<=0))
Dt <- na.omit(s$rts-s$Rb)
sharpe = (mean(Dt)*252)/(sd(Dt)*sqrt(252))
} else {
sharpe = -100
}
#have to return negative sharpe because DEoptim minimises the fitness function
return(-sharpe)
}

lower = c(4,4,4)
upper = c(63,63,63)

set.seed(1234)
parent=z

#perform optimizaztion, set value to reach at -6 (i.e. sharpe ratio of 6 in this example) and maximum interations = 100
outDEoptim = DEoptim(MACDFitness,lower,upper, DEoptim.control(VTR=-6,itermax=100,))
summary(outDEoptim)

__________________________________________________
Commonwealth Bank
Darko Roupell
Associate Quantitative Analyst
Institutional Banking & Markets
Equities Research
Darling Park Tower 1
Level 23, 201 Sussex Street
Sydney, NSW 2000
P: +61 2 9117 1254
F: +61 2 9118 1000
M: +61 400 170 515
E: [hidden email]
Our vision is to be Australia's finest financial services organisation through excelling in customer service.

Email Security
This email is sent solely for informational purposes. Hoax emails, commonly referred to as phishing, can appear to be from the Commonwealth Bank and ask you to update or confirm details such as client numbers, passwords, personal identification questions, contact details or account numbers. The Commonwealth Bank will never send you an email asking you to confirm, update or reveal your confidential banking information.
Important Information
Produced by Global Markets Research, a business unit of Commonwealth Bank of Australia ABN 48 123 123 124 - AFSL 234945 (Commonwealth Bank). This publication is based on information available at the time of publishing. We believe that the information in this communication is correct and any opinions, conclusions or recommendations are reasonably held or made as at the time of its compilation, but no warranty is made as to accuracy, reliability or completeness. To the extent permitted by law, neither Commonwealth Bank nor any of its subsidiaries accept liability to any person for loss or damage arising from the use of this communication. This communication does not purport to be a complete statement or summary.
The information provided has been prepared without considering your objectives, financial situation or needs, and before acting on the information, you should consider its appropriateness to your circumstances. No person should act on the basis of this report without considering and if necessary taking appropriate professional advice upon their own particular circumstances.
Commonwealth Bank of Australia, as a provider of investment, borrowing and other financial services undertakes financial transactions with many corporate entities in Australia. This may include any corporate issuer referred to in this communication. Commonwealth Bank and its subsidiaries have effected or may effect transactions for their own account in any investments or related investments referred to herein. In the case of certain securities Commonwealth Bank is or may be the only market maker.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Patrick Burns
Sent: Friday, 9 March 2012 6:24 AM
To: Daniel Cegiełka
Cc: [hidden email]
Subject: Re: [R-SIG-Finance] Are there genetic algorithm for trading strategy evolution in R?

Yes, serious chance of doing it poorly
without years of intense work.

On 08/03/2012 19:14, Daniel Cegiełka wrote:

--
Patrick Burns
[hidden email]
http://www.burns-stat.com
http://www.portfolioprobe.com/blog
twitter: @portfolioprobe

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

************** IMPORTANT MESSAGE *****************************
This e-mail message is intended only for the addressee(s) and contains information which may be
confidential.
If you are not the intended recipient please advise the sender by return email, do not use or
disclose the contents, and delete the message and any attachments from your system. Unless
specifically indicated, this email does not constitute formal advice or commitment by the sender
or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
We can be contacted through our web site: commbank.com.au.
If you no longer wish to receive commercial electronic messages from us, please reply to this
e-mail by typing Unsubscribe in the subject line.
**************************************************************

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

LosemindL

Re: Are there genetic algorithm for trading strategy evolution in R?

Thanks so much Darko!

It's really a great contribution to the thread and to the entire R finance
community!

We really appreciate your kindness!

I will study deep into your example and digest it further and I will
consult with your expertise with more questions.

Thanks so very much again!

On Thu, Mar 8, 2012 at 4:23 PM, Roupell, Darko <[hidden email]>wrote:

> To contribute to this discussion with a more concrete example of genetic
> algorithm usage for trading purpose I have disclosed simple code snippet
> how to implement it using DEoptim. GALGO is alternative to DEoptim in R.
>
> Micheal, generally I do not provide complete examples so for this
> exception thank to Patrick Burns as he helped me in the past with
> PortfolioProbe Optimizer. Hope it helps.
>
>
> ******Disclaimer this is just example for learning purposes but no
> warranty is made as to accuracy and no liability is accepted if used for
> commercial purposes****
>
> require(quantmod)
> require(PerformanceAnalytics)
> require(blotter)
> require(DEoptim)
> ################ MACD TEST
> z=read.csv(file="G:\\QERD\\GALGO\\first.csv",header=TRUE,sep=",",stringsAsFactors=FALSE,
> colClasses=c('character','numeric','numeric'))
> z =as.xts(z, as.POSIXct(z$Date))
> #force columns into numeric values
> z$sum=as.numeric(z$INTC)+as.numeric(z$IEF)
> z = z[,c("INTC", "IEF")]
> # Let's think about returns instead of prices...
> # Ra is the log return for a buy-and-hold strategy, Rb the 'benchmark.'
> # We will extensively use these log-return series in the sequel.
>
> z$Ra = Return.calculate(z$INTC)
> z$Rb = Return.calculate(z$IEF)
> # we will optimise over in sample data
> insample=z['2008:']
>
> #the fitness function must obtain the data
> MACDFitness <- function(params) {
> fast <- params[1]
> slow <- params[2]
> sig <- params[3]
> if ((fast <= slow - 1) & (slow >= 4) & (fast >=4) & ((sig >= fast) &
> (sig <= slow))) { #certain conditions does not make sense
> params = paste("F: ",fast," slow: ",slow,sep="")
> x <- MACD(parent$INTC, nFast=fast, nSlow=slow,
> nSig=sig,maType="EMA")
> position <- sign(x[,1]-x[,2])
> s <- xts(position,order.by=index(parent))
> s$Ra <- parent$Ra
> s$Rb <- parent$Rb
> s$rts <- (s$Ra*(s$macd>0)) + (s$Rb*(s$macd<=0))
> Dt <- na.omit(s$rts-s$Rb)
> sharpe = (mean(Dt)*252)/(sd(Dt)*sqrt(252))
> } else {
> sharpe = -100
> }
> #have to return negative sharpe because DEoptim minimises the fitness
> function
> return(-sharpe)
> }
>
>
> lower = c(4,4,4)
> upper = c(63,63,63)
>
> set.seed(1234)
> parent=z
>
> #perform optimizaztion, set value to reach at -6 (i.e. sharpe ratio of 6
> in this example) and maximum interations = 100
> outDEoptim = DEoptim(MACDFitness,lower,upper,
> DEoptim.control(VTR=-6,itermax=100,))
> summary(outDEoptim)
>
> __________________________________________________
> Commonwealth Bank
> Darko Roupell
> Associate Quantitative Analyst
> Institutional Banking & Markets
> Equities Research
> Darling Park Tower 1
> Level 23, 201 Sussex Street
> Sydney, NSW 2000
> P: +61 2 9117 1254
> F: +61 2 9118 1000
> M: +61 400 170 515
> E: [hidden email]
> Our vision is to be Australia's finest financial services organisation
> through excelling in customer service.
>
> Email Security
> This email is sent solely for informational purposes. Hoax emails,
> commonly referred to as phishing, can appear to be from the Commonwealth
> Bank and ask you to update or confirm details such as client numbers,
> passwords, personal identification questions, contact details or account
> numbers. The Commonwealth Bank will never send you an email asking you to
> confirm, update or reveal your confidential banking information.
> Important Information
> Produced by Global Markets Research, a business unit of Commonwealth Bank
> of Australia ABN 48 123 123 124 - AFSL 234945 (Commonwealth Bank). This
> publication is based on information available at the time of publishing.
> We believe that the information in this communication is correct and any
> opinions, conclusions or recommendations are reasonably held or made as at
> the time of its compilation, but no warranty is made as to accuracy,
> reliability or completeness. To the extent permitted by law, neither
> Commonwealth Bank nor any of its subsidiaries accept liability to any
> person for loss or damage arising from the use of this communication. This
> communication does not purport to be a complete statement or summary.
> The information provided has been prepared without considering your
> objectives, financial situation or needs, and before acting on the
> information, you should consider its appropriateness to your circumstances.
> No person should act on the basis of this report without considering and if
> necessary taking appropriate professional advice upon their own particular
> circumstances.
> Commonwealth Bank of Australia, as a provider of investment, borrowing and
> other financial services undertakes financial transactions with many
> corporate entities in Australia. This may include any corporate issuer
> referred to in this communication. Commonwealth Bank and its subsidiaries
> have effected or may effect transactions for their own account in any
> investments or related investments referred to herein. In the case of
> certain securities Commonwealth Bank is or may be the only market maker.
>
>
> -----Original Message-----
> From: [hidden email] [mailto:
> [hidden email]] On Behalf Of Patrick Burns
> Sent: Friday, 9 March 2012 6:24 AM
> To: Daniel CegieÅka
> Cc: [hidden email]
> Subject: Re: [R-SIG-Finance] Are there genetic algorithm for trading
> strategy evolution in R?
>
> Yes, serious chance of doing it poorly
> without years of intense work.
>
> On 08/03/2012 19:14, Daniel CegieÅka wrote:
> > Let me add my two cents. Old Max Dama blog (mirror):
> >
> >
> http://smartdatacollective.com/maxdama/22571/voodoo-spectrum-machine-learning-and-data-sets
> >
> >
> > Optimization is good to examine the sensitivity of the model and the
> > selection of appropriate parameters - this is useful. But playing with
> > the evolutionary strategy, what you Michael ask, is very risky.
> >
> > regards,
> > Daniel
> >
> >
> >
> > 2012/3/8 Patrick Burns <[hidden email]
> > <mailto:[hidden email]>>
> >
> > Comments inline.
> >
> >
> > On 08/03/2012 18:16, Michael wrote:
> >
> > Thanks folks!
> >
> > After digging further on the Internet, I have the following
> > questions:
> >
> > Q1: I read the following article:
> >
> >
> http://cran.r-project.org/web/__packages/DEoptim/vignettes/__DEoptimPortfolioOptimization.__pdf
> > <
> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
> >
> >
> > It seems that there are a bunch of parameters in this optimizer
> > and the
> > results are sensitive to these parameters.
> >
> > So there is another layer of optimization with respect to these
> > optimizer
> > parameters.
> >
> > Is the "tweaking" of these optimizer parameters data-mining,
> > which will
> > lead to data-snooping bias?
> >
> >
> > I wouldn't think so, but there might be
> > a way to manage it.
> >
> >
> >
> > Q2: Due to the random nature of the optimizer, each time you run
> the
> > backtest, you will have different performance.
> >
> > What do you do in that case?
> >
> >
> > That may be a good thing, if you are
> > willing to use it.
> >
> > In what I've done on backtesting:
> >
> >
> http://www.portfolioprobe.com/__2010/11/05/backtesting-almost-__wordless/
> > <
> http://www.portfolioprobe.com/2010/11/05/backtesting-almost-wordless/>
> >
> > I show how to assess whether the strategy
> > is better than luck by using random trades.
> >
> > The standard thing to assume (as I do in
> > that piece) is that the optimization is
> > noiseless. But really the optimization
> > depends on a multitude of subtle influences.
> > Even if you always got the exact global
> > optimum, if a variance or expected return
> > were slightly different, you could get a
> > very different path. The "optimal" path
> > is fuzzy in actuality.
> >
> >
> >
> > So for out-of-sample real-trading, we are trading a random
> strategy?
> >
> >
> > Yes. But the inputs are random so even
> > non-stochastic optimizers give you a
> > random strategy in a sense.
> >
> >
> >
> > Q3: It's pretty easy to understand using Genetic Algorithms to
> > serve as a
> > replacement for regular optimizers;
> >
> > but using Genetic Algorithms to evolve trading strategies seem
> to be
> > different. Anywhere we could find such an example in R?
> >
> >
> > Yes, that is different.
> >
> > In
> > https://stat.ethz.ch/__pipermail/r-sig-finance/__2010q4/007033.html
> > <https://stat.ethz.ch/pipermail/r-sig-finance/2010q4/007033.html>
> > you can find Josh quoting me quoting Lao-Tzu
> > on why you are unlikely to find much useful
> > on that subject.
> >
> > Pat
> >
> >
> >
> >
> >
> >
> > On Thu, Mar 8, 2012 at 8:25 AM, Zachary
> > Mayer<[hidden email] <mailto:[hidden email]>>
> wrote:
> >
> > There is the
> > DEoptim<http://cran.r-project.
> __org/web/packages/DEoptim/__index.html
> > <http://cran.r-project.org/web/packages/DEoptim/index.html
> >>library
> > in r, which is an excellent library for differential
> > evolution. If
> > you can define your trading strategy in terms of a bunch of
> > parameters to
> > adjust and an objective function (i.e. turn it into an
> > optimization
> > problem), DEoptim will help you find the minimum (or
> maximum).
> >
> > DEoptim works well on non-differentiable problems with many
> > local minima.
> > Here is an example of using it to solve a portfolio
> > optimization problem:
> >
> >
> http://cran.r-project.org/web/__packages/DEoptim/vignettes/__DEoptimPortfolioOptimization.__pdf
> > <
> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
> >
> >
> >
> >
> > On Thu, Mar 8, 2012 at 12:43 AM, Sofian
> > Hadiwijaya<reztinpeace@gmail.__com
> > <mailto:[hidden email]>>wrote:
> >
> > how about quantmod library..
> >
> > On Wed, Mar 7, 2012 at 10:30 PM,
> > Michael<[hidden email]
> > <mailto:[hidden email]>> wrote:
> >
> > Hi all, Good morning, good afternoon and good
> evening!
> >
> > Could anybody please kindly point me to resources in
> > R which shows about
> > how to use Genetic algorithm to evolve trading
> > strategies?
> >
> > I did a lot search on Google these days and
> > certainly it's a
> >
> > well-covered
> >
> > and popular topic, but I don't see anywhere in R...
> >
> > Thanks a lot!
> >
> > [[alternative HTML version deleted]]
> >
> > _________________________________________________
> > [hidden email]
> > <mailto:[hidden email]> mailing list
> >
> https://stat.ethz.ch/mailman/__listinfo/r-sig-finance <
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> > -- Subscriber-posting only. If you want to post,
> > subscribe first.
> > -- Also note that this is not the r-help list where
> > general R questions
> > should go.
> >
> >
> > [[alternative HTML version deleted]]
> >
> > _________________________________________________
> > [hidden email]
> > <mailto:[hidden email]> mailing list
> > https://stat.ethz.ch/mailman/__listinfo/r-sig-finance
> > <https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> > -- Subscriber-posting only. If you want to post,
> > subscribe first.
> > -- Also note that this is not the r-help list where
> > general R questions
> > should go.
> >
> >
> >
> >
> > [[alternative HTML version deleted]]
> >
> > _________________________________________________
> > [hidden email] <mailto:[hidden email]>
> > mailing list
> > https://stat.ethz.ch/mailman/__listinfo/r-sig-finance
> > <https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R
> > questions should go.
> >
> >
> > --
> > Patrick Burns
> > [hidden email] <mailto:[hidden email]>
> > http://www.burns-stat.com
> > http://www.portfolioprobe.com/__blog
> > <http://www.portfolioprobe.com/blog>
> > twitter: @portfolioprobe
> >
> >
> > _________________________________________________
> > [hidden email] <mailto:[hidden email]>
> > mailing list
> > https://stat.ethz.ch/mailman/__listinfo/r-sig-finance
> > <https://stat.ethz.ch/mailman/listinfo/r-sig-finance>
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R
> > questions should go.
> >
> >
>
> --
> Patrick Burns
> [hidden email]
> http://www.burns-stat.com
> http://www.portfolioprobe.com/blog
> twitter: @portfolioprobe
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>
> ************** IMPORTANT MESSAGE *****************************
> This e-mail message is intended only for the addressee(s) and contains
> information which may be
> confidential.
> If you are not the intended recipient please advise the sender by return
> email, do not use or
> disclose the contents, and delete the message and any attachments from
> your system. Unless
> specifically indicated, this email does not constitute formal advice or
> commitment by the sender
> or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its
> subsidiaries.
> We can be contacted through our web site: commbank.com.au.
> If you no longer wish to receive commercial electronic messages from us,
> please reply to this
> e-mail by typing Unsubscribe in the subject line.
> **************************************************************
>
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

LosemindL

Re: Are there genetic algorithm for trading strategy evolution in R?

I have a quick question regarding all these "iterative" methods though:

Genetic algo is an iterative optimizer and the number of iterations is a
parameter.

Many similar algos exist. For example, neural networks,
clustering(K-means), etc.

Even the simplest one - the robust linear regression is an iterative algo.

On the stock data I have, running robust linear regression using "rlm" in R
often gives warnings on failure to converge within 20 steps.

What do you do in such situation?

I tried changing the "20" steps to "200" steps... and of course the
warnings were reduced... and the overall Sharpe ratio got some
improvement... but when I tried changing it to "100" steps, the overall
Sharpe ratio was actually worse.

Therefore, my question is, how do you deal with parameters in such tools,
be it robust linear regression, neural networks, clustering, or our topic
today, the stochastic optimizer the Genetic algo...?

Is tweaking these parameters date-mining?

Do optimized versions of such parameters have out-sample stability?

On the other hand, maybe I am just overly-fearing about the danger of
data-mining?

What do you think?

Thanks a lot!

[[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

Enrico Schumann

Re: Are there genetic algorithm for trading strategy evolution in R?

In reply to this post by LosemindL

Am 08.03.2012 19:16, schrieb Michael:

> Thanks folks!
>
> After digging further on the Internet, I have the following
> questions:
>
> Q1: I read the following article:
>
> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
>
> It seems that there are a bunch of parameters in this optimizer and
> the results are sensitive to these parameters.
>
> So there is another layer of optimization with respect to these
> optimizer parameters.
>
> Is the "tweaking" of these optimizer parameters data-mining, which
> will lead to data-snooping bias?

No. You need to distinguish between your optimisation model on the one
hand, and the numerical technique you use to solve the model on the
other. And Differential Evolution (DE) is a numerical technique. If your
model overfits, it is because of the model, not the optimisation
technique. Suppose you wanted to estimate a linear regression, with the
mean squared residual as the criterion of fit. Now you can compute a
solution via QR; or if you use DE *properly*. it will give you exactly
the same fit (up to numerical precision). Whether the model overfits,
depends on how you set up the model, how you select the data in the
model -- it has nothing to do with the numerical technique.

>
> Q2: Due to the random nature of the optimizer, each time you run the
> backtest, you will have different performance.
>
> What do you do in that case?

There is only one way to find out: run experiments. Put an outer loop
around your backtest in which repeat your analysis; then you can see how
the stochastic nature of the optimisation affects your results. In my
view, it's not a problem: see for instance this paper

http://ssrn.com/abstract=1420058

And experiments are also the only reliable way to find out if your
optimisation technique works properly. All described in detail in this
book (of which I happen to be a co-author)

@BOOK{Gilli2011b,
title = {Numerical Methods and Optimization in Finance},
publisher = {Academic Press},
year = {2011},
author = {Gilli, Manfred and Maringer, Dietmar and Schumann, Enrico}
}

>
> So for out-of-sample real-trading, we are trading a random strategy?

No, again: model vs optimisation. Eventually, you have to accept some
set of parameters for your optimisation and use these to trade.

>
> Q3: It's pretty easy to understand using Genetic Algorithms to serve
> as a replacement for regular optimizers;
>
> but using Genetic Algorithms to evolve trading strategies seem to be
> different. Anywhere we could find such an example in R?
>

What you probably mean is Genetic Programming, not Genetic Algorithms.

Regards,
Enrico

--
Enrico Schumann
Lucerne, Switzerland
http://nmof.net

>
>
>
> On Thu, Mar 8, 2012 at 8:25 AM, Zachary Mayer<[hidden email]>
> wrote:
>
>> There is the
>> DEoptim<http://cran.r-project.org/web/packages/DEoptim/index.html>library
>> in r, which is an excellent library for differential evolution.
>> If you can define your trading strategy in terms of a bunch of
>> parameters to adjust and an objective function (i.e. turn it into
>> an optimization problem), DEoptim will help you find the minimum
>> (or maximum).
>>
>> DEoptim works well on non-differentiable problems with many local
>> minima. Here is an example of using it to solve a portfolio
>> optimization problem:
>>
>> http://cran.r-project.org/web/packages/DEoptim/vignettes/DEoptimPortfolioOptimization.pdf
>>
>>
>>
>>
>>

On Thu, Mar 8, 2012 at 12:43 AM, Sofian
Hadiwijaya<[hidden email]>wrote:

>>
>>> how about quantmod library..
>>>
>>> On Wed, Mar 7, 2012 at 10:30 PM, Michael<[hidden email]>
>>> wrote:
>>>
>>>> Hi all, Good morning, good afternoon and good evening!
>>>>
>>>> Could anybody please kindly point me to resources in R which
>>>> shows about how to use Genetic algorithm to evolve trading
>>>> strategies?
>>>>
>>>> I did a lot search on Google these days and certainly it's a
>>> well-covered
>>>> and popular topic, but I don't see anywhere in R...
>>>>
>>>> Thanks a lot!
>>>>
>

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

LosemindL

Re: Are there genetic algorithm for trading strategy evolution in R?

In reply to this post by darko

Hi Darko,

Thanks so much for your excellent contribution.

I have spent some time studying your program and play around with the
packages...

May I consult you with a few questions?

1. What does the following line in your code mean?

s$rts <- (s$Ra*(s$macd>0)) + (s$Rb*(s$macd<=0))

2. I saw you are using Genetic Algo as a replacement for the optimizer... I
guess my original question was about

how to evolve the strategy parameters over time? So it's a dynamic
multi-period problem. For example, in this MACD example,

if you use a grid search to search for the optimal slow/fast parameters,
you will probably do better than using the

Genetic Algo. On a parallel computing platform, this is also feasible and
fast.

But if you extend this problem into a multi-period dynamic problem - at
each period, you always pick the best performer based

on past performance using either grid search or Genetic Algo, you will find
the overall out-of-sample in the each next period to be poor...

Therefore, it seems that Genetic Algo is just a replacement of the
optimizer and it's going to be useful in high-dimensional search...

But it doesn't give "staying power" to the selected model...

Am I right?

Thanks al ot!

On Thu, Mar 8, 2012 at 4:23 PM, Roupell, Darko <[hidden email]>wrote:

Ulrich Staudinger-2

Re: Are there genetic algorithm for trading strategy evolution in R?

On Tue, Mar 13, 2012 at 9:20 PM, Michael <[hidden email]> wrote:
> on past performance using either grid search or Genetic Algo, you will
find
the overall out-of-sample in the each next period to be poor...

see comments below.

>Therefore, it seems that Genetic Algo is just a replacement of the
optimizer and it's going to be useful in high-dimensional search...

> But it doesn't give "staying power" to the selected model..

The "staying power", or rather stability, has nothing to do with the
optimization algo, but rather with the underlying problem space and your
space descriptor. If your problem space is unstable your space descriptor
should be able to cover this.

I wouldn't declare GA or any other optimization technique as a tool usable
for one type of problem only ..

As Enrico already said, it's not the fault of the optimization technique,
but rather the fault of the model.

[.. snipping 431 lines of quoted text ...]

--
Ulrich Staudinger

http://www.activequant.org
Connect online: https://www.xing.com/profile/Ulrich_Staudinger

[[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.