R › Rmetrics

error in yang.zhang volatility{TTR}

Classic

List

Threaded

6 messages Options

J Toll

error in yang.zhang volatility{TTR}

Hi,

I've been going through the code for the yang.zhang calculation method
of volatility in the TTR package and believe I've found the same error
as the one from last year on the Close to Close volatility estimator.

https://stat.ethz.ch/pipermail/r-sig-finance/2011q2/007989.html

Like the previous error, one of the first clues that there was a
problem was an excessive number of NA values generated at the
beginning of the output (e.g. 40 NA's for n = 20).

I believe the errors are in these two calculations:

s2o <- N/(n - 1) * runSum((log(OHLC[, 1]/Cl1) - 1/n *
runSum(log(OHLC[, 1]/Cl1), n))^2, n)
s2c <- N/(n - 1) * runSum((log(OHLC[, 4]/OHLC[, 1]) -
1/n * runSum(log(OHLC[, 4]/OHLC[, 1]), n))^2, n)

Basically, s2o is just supposed to be the variance of the normalized open,
log(OHLC[, 1]/Cl1

and s2c is the variance of the normalized close.
log(OHLC[, 4]/OHLC[, 1]

The problem in both formula is, as in Close to Close, that you have
two nested runSum calculations being used to calculate variance. For
example, if n = 20, the first 20 values should be differenced against
the same mean. That's not happening here. The first value is being
differenced against the first mean, the second value is being
differenced against the second mean, and so on.

As a simple fix, I would suggest these corrections:

s2o <- N * runVar(log(OHLC[, 1] / Cl1), n)
s2c <- N * runVar(log(OHLC[, 4] / OHLC[, 1]), n)

I would welcome the input of someone with reference data that could
double-check this against a known value.

Thanks,

James

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

J Toll

Re: error in yang.zhang volatility{TTR}

On Mon, May 21, 2012 at 11:32 AM, J Toll <[hidden email]> wrote:

> Hi,
>
> I've been going through the code for the yang.zhang calculation method
> of volatility in the TTR package and believe I've found the same error
> as the one from last year on the Close to Close volatility estimator.
>
> https://stat.ethz.ch/pipermail/r-sig-finance/2011q2/007989.html
>
> Like the previous error, one of the first clues that there was a
> problem was an excessive number of NA values generated at the
> beginning of the output (e.g. 40 NA's for n = 20).
>
> I believe the errors are in these two calculations:
>
> s2o <- N/(n - 1) * runSum((log(OHLC[, 1]/Cl1) - 1/n *
> runSum(log(OHLC[, 1]/Cl1), n))^2, n)
> s2c <- N/(n - 1) * runSum((log(OHLC[, 4]/OHLC[, 1]) -
> 1/n * runSum(log(OHLC[, 4]/OHLC[, 1]), n))^2, n)
>
> Basically, s2o is just supposed to be the variance of the normalized open,
> log(OHLC[, 1]/Cl1
>
> and s2c is the variance of the normalized close.
> log(OHLC[, 4]/OHLC[, 1]
>
> The problem in both formula is, as in Close to Close, that you have
> two nested runSum calculations being used to calculate variance. For
> example, if n = 20, the first 20 values should be differenced against
> the same mean. That's not happening here. The first value is being
> differenced against the first mean, the second value is being
> differenced against the second mean, and so on.
>
> As a simple fix, I would suggest these corrections:
>
> s2o <- N * runVar(log(OHLC[, 1] / Cl1), n)
> s2c <- N * runVar(log(OHLC[, 4] / OHLC[, 1]), n)

Sorry, to get that to work, it should actually be:

s2o <- N * runVar(log(OHLC[, 1] / Cl1), n = n)
s2c <- N * runVar(log(OHLC[, 4] / OHLC[, 1]), n = n)

It needs to be "n = n", otherwise runVar confuses n for y.

James

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

Hideyoshi Maeda

Re: error in yang.zhang volatility{TTR}

In reply to this post by J Toll

Hi,

Doing a quick comparison of the difference in code for the volatility estimate in using 'yang.zhang' as a calculation method, on S&P 500 data, going back from the beginning of 2001 till today (22 May 2012). These were the results from the following lines of code:

N.B. volatility.new is the edited code with the suggestion of using runVar instead of runSum, as suggested by James

N.B SPX.ohlc is an xts object of that contains OHLC data for the Bloomberg ticker "SPX Index"

> original.vol.code <- volatility(SPX.ohlc, calc="yang.zhang")

> new.vol.code <- volatility.new(SPX.ohlc, calc="yang.zhang")

> plot((new.vol.code-original.vol.code)*100/original.vol.code, main="% difference in volatility esitimates

+ when comparing new and old code")

Some quite significant percentage differences getting up to approx. -10% viewing it as a histogram in terms of differences, you get...

> hist((new.vol.code-original.vol.code)*100/original.vol.code, breaks="Scott", main="Histogram of the same data")

Hopefully that gives some indication of the differences...

Hideyoshi

-------- Original Message --------
Subject:
Re: [R-SIG-Finance] error in yang.zhang volatility{TTR}
Date:
Mon, 21 May 2012 12:40:29 -0500
From:
J Toll [hidden email]
To:
[hidden email]
On Mon, May 21, 2012 at 11:32 AM, J Toll [hidden email] wrote:
> Hi,
> 
> I've been going through the code for the yang.zhang calculation method
> of volatility in the TTR package and believe I've found the same error
> as the one from last year on the Close to Close volatility estimator.
> 
> https://stat.ethz.ch/pipermail/r-sig-finance/2011q2/007989.html
> 
> Like the previous error, one of the first clues that there was a
> problem was an excessive number of NA values generated at the
> beginning of the output (e.g. 40 NA's for n = 20).
> 
> I believe the errors are in these two calculations:
> 
>        s2o <- N/(n - 1) * runSum((log(OHLC[, 1]/Cl1) - 1/n *
>            runSum(log(OHLC[, 1]/Cl1), n))^2, n)
>        s2c <- N/(n - 1) * runSum((log(OHLC[, 4]/OHLC[, 1]) -
>            1/n * runSum(log(OHLC[, 4]/OHLC[, 1]), n))^2, n)
> 
> Basically, s2o is just supposed to be the variance of the normalized open,
>    log(OHLC[, 1]/Cl1
> 
> and s2c is the variance of the normalized close.
>    log(OHLC[, 4]/OHLC[, 1]
> 
> The problem in both formula is, as in Close to Close, that you have
> two nested runSum calculations being used to calculate variance.  For
> example, if n = 20, the first 20 values should be differenced against
> the same mean.  That's not happening here.  The first value is being
> differenced against the first mean, the second value is being
> differenced against the second mean, and so on.
> 
> As a simple fix, I would suggest these corrections:
> 
>        s2o <- N * runVar(log(OHLC[, 1] / Cl1), n)
>        s2c <- N * runVar(log(OHLC[, 4] / OHLC[, 1]), n)
 
 
Sorry, to get that to work, it should actually be:
 
        s2o <- N * runVar(log(OHLC[, 1] / Cl1), n = n)
        s2c <- N * runVar(log(OHLC[, 4] / OHLC[, 1]), n = n)
 
It needs to be "n = n", otherwise runVar confuses n for y.
 
 
James
 
_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
This email is confidential. If you have received it in error, please destroy this email and any/all attached documents immediately. Please also notify the sender. This message is not a solicitation and, without express information to the contrary, does not take into account the investment objectives (or otherwise) of each client, or prospective client. The information herein is for the private information of the recipient and is not to be construed as an offer to sell or solicitation to buy any interest in particular funds or securities which are referred to herein. Any simulated historic/statistical analyses that are provided in connection with explanations of the potential returns of the Product use simulated analysis and hypothetical circumstances to estimate how the Product may have performed prior to their actual existence. Neither the provision nor the content of this email creates any obligations or liability of White & Co. towards the users. White & Co. does not make any representations, warranties, express or implied, as to the quality, accuracy, timeliness, continued availability or completeness of the information contained herein, including, but not limited to, the contents of any private placement memorandum or fund fact sheet. White & Co. accepts no liability relating to direct or indirect losses, immaterial damage or consequential damage including loss of profit (even where White & Co. has been made aware of the possibility of such damages). This limitation of liability also holds for directors and employees of White & Co. The information in this email has been prepared on information deemed to be reliable. White & Co. Pty Ltd does not represent that this material is accurate, complete or current, and accepts no liability if it is not. White & Co, its subsidiaries and partners may have positions in - and buy or sell - any of the securities mentioned in this e-mail without notice. White & Co. Pty. Ltd. is regulated by the CFTC and ASIC.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

J Toll

Re: error in yang.zhang volatility{TTR}

On Mon, May 21, 2012 at 9:20 PM, Hideyoshi Maeda
<[hidden email]>wrote:

> Doing a quick comparison of the difference in code for the volatility
> estimate in using 'yang.zhang' as a calculation method,
>
on S&P 500 data, going back from the beginning of 2001 till today (22 May
2012). These were the results from the following lines of code:

>
> N.B. volatility.new is the edited code with the suggestion of using runVar
> instead of runSum, as suggested by James
> N.B SPX.ohlc is an xts object of that contains OHLC data for the Bloomberg
> ticker "SPX Index"
>
> > original.vol.code <- volatility(SPX.ohlc, calc="yang.zhang")
> > new.vol.code <- volatility.new(SPX.ohlc, calc="yang.zhang")
> > plot((new.vol.code-original.vol.code)*100/original.vol.code, main="%
> difference in volatility esitimates
> + when comparing new and old code")
>
> Some quite significant percentage differences getting up to approx. -10%
> viewing it as a histogram in terms of differences, you get...
>

I was actually surprised that the calculation isn't off by more, but the
inner runSum which calculates a rolling mean is usually fairly stable,
especially for longer n. I don't know if Bloomberg can calculate
volatility via yang.zhang, but that would be a worthwhile reference to
compare.

Best,

James

[[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

Joshua Ulrich

Re: error in yang.zhang volatility{TTR}

In reply to this post by J Toll

Hi James,

On Mon, May 21, 2012 at 12:40 PM, J Toll <[hidden email]> wrote:

> On Mon, May 21, 2012 at 11:32 AM, J Toll <[hidden email]> wrote:
>> Hi,
>>
>> I've been going through the code for the yang.zhang calculation method
>> of volatility in the TTR package and believe I've found the same error
>> as the one from last year on the Close to Close volatility estimator.
>>
>> https://stat.ethz.ch/pipermail/r-sig-finance/2011q2/007989.html
>>
>> Like the previous error, one of the first clues that there was a
>> problem was an excessive number of NA values generated at the
>> beginning of the output (e.g. 40 NA's for n = 20).
>>
>> I believe the errors are in these two calculations:
>>
>> s2o <- N/(n - 1) * runSum((log(OHLC[, 1]/Cl1) - 1/n *
>> runSum(log(OHLC[, 1]/Cl1), n))^2, n)
>> s2c <- N/(n - 1) * runSum((log(OHLC[, 4]/OHLC[, 1]) -
>> 1/n * runSum(log(OHLC[, 4]/OHLC[, 1]), n))^2, n)
>>

<snip>

>>
>> As a simple fix, I would suggest these corrections:
>>
>> s2o <- N * runVar(log(OHLC[, 1] / Cl1), n)
>> s2c <- N * runVar(log(OHLC[, 4] / OHLC[, 1]), n)
>
>
> Sorry, to get that to work, it should actually be:
>
> s2o <- N * runVar(log(OHLC[, 1] / Cl1), n = n)
> s2c <- N * runVar(log(OHLC[, 4] / OHLC[, 1]), n = n)
>
> It needs to be "n = n", otherwise runVar confuses n for y.
>
>
> James
>

Thank you for another very helpful patch. From looking at the svn
log, it looks like I got more things wrong than right with this
calculation. :)

I also added the ability for users to specify alpha via '...' (the
default value is 1.34).

Best,
--
Joshua Ulrich | FOSS Trading: www.fosstrading.com

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

J Toll

Re: error in yang.zhang volatility{TTR}

On Tue, May 29, 2012 at 10:08 PM, Joshua Ulrich <[hidden email]> wrote:

>> Sorry, to get that to work, it should actually be:
>>
>> s2o <- N * runVar(log(OHLC[, 1] / Cl1), n = n)
>> s2c <- N * runVar(log(OHLC[, 4] / OHLC[, 1]), n = n)
>>
>> It needs to be "n = n", otherwise runVar confuses n for y.
>>
> Thank you for another very helpful patch. From looking at the svn
> log, it looks like I got more things wrong than right with this
> calculation. :)
>
> I also added the ability for users to specify alpha via '...' (the
> default value is 1.34).
>
> Best,
> --
> Joshua Ulrich | FOSS Trading: www.fosstrading.com

Joshua,

Thanks for taking a look at that. I know you were joking about
getting "more things wrong than right", but I think I partially melted
my brain thinking through the vectorization. One stand-alone
calculation may be fairly easy, but then vectorizing the whole thing
can be tricky.

I think the patches all look good on r-forge. Thanks again.

Best,

James

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.