Re: Fwd: Re: [Fwd: Performance Analytics internal multivariateMoments calculations]

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Re: [Fwd: Performance Analytics internal multivariateMoments calculations]

Rmetrics mailing list
Gentlemen,

I so appreciate this feedback.  I do have a followup just so I fully
understand.  The functions mVaR.MM and mES.MM both require the raw 3rd
and 4th moments, not the standardized skewness and kurtosis statistics.  
And, would Dries mind sharing the simplified formulas that he mentions.

Again, thank you both very much.


Joe

On 05/23/2017 05:14 AM, Brian G. Peterson wrote:

>
>
>
> -------- Forwarded Message --------
> Subject:     Re: [Fwd: [R-SIG-Finance] Performance Analytics internal
> multivariateMoments calculations]
> Date:     Tue, 23 May 2017 02:10:21 +0000
> From:     Dries Cornilly <[hidden email]>
> To:     Brian G. Peterson <[hidden email]>
>
>
>
> Also, Joe Byers did not run in the same issue. He is confusing the
> kurtosis and the excess kurtosis.
>
> The function “kurtosis.MM” in “mVaR.MM” does not return the excess
> kurtosis, but rather m4 / sd^4 and hence the -3 still needs to be
> done. His example where he claims that GVaR and MVaR should return the
> same is not correct. He uses 0 as fourth order moment in the input,
> which is only valid for a degenerate distribution. The input for
> equality should rather be
> GVaR(w, Mean, Stdev, .95)
> MVaR(w, Mean, Stdev, 0, 3 * Stdev^4, .95) # note the difference in
> fourth moment.
> For a Gaussian the excess kurtosis is zero, or equivalently, the
> standardised fourth moment (m4 / sd^4) is equal to 3, but the fourth
> moment of a Gaussian is not equal to zero, it is equal to a function
> of the variance.
>
> Additionally, he will generate problems when using the skewness input
> as in his ‘corrected’ version. The function “mVaR.MM” takes the raw
> third order central moment(s) and standardises in the function itself,
> whereas his altered function will standardise again on the already
> standardised input and hence give a wrong result.
>
> Regards
> Dries
>
>
> On 22 May 2017 at 20:32:04, Dries Cornilly ([hidden email]
> <mailto:[hidden email]>) wrote:
>
>> I showed it to Joshua this morning and it seemed like a pure integer
>> problem.
>>
>> library(PerformanceAnalytics)
>> X <- matrix(1:12, ncol = 3)
>> M3.MM(X) # this one behaves strangely (at this time, X is still filled
>> with integers)
>> M3.MM(X * 1.0)  # multiplying by 1.0 to cast to double provides a zero
>> coskewness matrix at it should
>>
>> However, I do have some code replicating the modified VaR and modified
>> Expected shortfall using simplified formulas (coming from a working
>> paper of Doug Martin if I recall correctly). I was going to suggest to
>> replace it since the output is identical and it seems more stable to
>> compute. When I have the alternative moment estimators using the
>> unique elements, it might be a good idea to replace the inside of mVaR
>> and mES using the multivariate moments to work with the vector of
>> unique elements (and extract the unique elements if the full matrices
>> are given). This will be a good memory improvement, especially when
>> computing the portfolio moments for mVaR and mES and the derivatives
>> needed for the component VaR and component ES.
>>
>>
>> Regards
>> Dries
>>
>>
>> On 22 May 2017 at 16:20:22, Brian G. Peterson ([hidden email]
>> <mailto:[hidden email]>) wrote:
>>
>>> Looks like Joe Byers ran into the same issue you found earlier today.
>>>
>>> --
>>> Brian G. Peterson
>>> http://braverock.com/brian/
>>> Ph: 773-459-4973
>>> IM: bgpbraverock
>>>
>>> -------- Forwarded Message --------
>>> From: Joe W. Byers via R-SIG-Finance <[hidden email]>
>>> Reply-to: "Joe W. Byers" <[hidden email]>
>>> To: [hidden email]
>>> Subject: [R-SIG-Finance] Performance Analytics internal
>>> multivariateMoments calculations
>>> Date: Mon, 22 May 2017 15:59:59 -0400
>>>
>>> All,
>>>
>>>
>>> I have carved on all methods in MultivariateMoments.R so I can all
>>> them
>>> directly
>>>
>>> I am working on the modified var calculations.  I ran the mVaR.MM on
>>> my
>>> data and the results were odd.  I reran setting skewness and kurtosis
>>> to
>>> 0 to compare with GVAR.MM.  Still questions.  I have the following
>>> example
>>>
>>> #test for sigfinance
>>> w = 1000000;
>>> Mean = 0.0001898251;
>>> Stdev = 0.01612464;
>>> ExKurtosis = 3.946156;
>>> Skewness = -0.1373454;
>>>
>>> GVaR = GVaR.MM(w,Mean,Stdev, .95)
>>> GVaR
>>> MVaR = mVaR.MM(w,Mean,Stdev, 0,0, .95);
>>> #shoud be equal to GVaR
>>> GVaR==MVaR
>>> MVaR
>>>
>>> mVaR.MM does not return the GVaR.MM.  I found the exkurt was not zero
>>> as
>>> I think it should be.  I remove the - 3 from that line and exkurt
>>> became
>>> zero and mVaR.MM == GVaR results.  I have included this modified
>>> version
>>> of mVaR.MM below and continued the test.  Is this an issue or am I
>>> missing something?
>>>
>>>
>>> #corrected exkurt calc
>>> mVaR.MM1 = function(w, mu, sigma, M3, M4, p ){
>>>    skew = skewness.MM(w,sigma,M3);
>>>    exkurt = kurtosis.MM(w,sigma,M4); #removed -3
>>>    z = qnorm(1-p);
>>>    zc = z + (1/6)*(z^2 -1)*skew
>>>    Zcf = zc + (1/24)*(z^3 - 3*z)*exkurt - (1/36)*(2*z^3 - 5*z)*skew^2;
>>>    return ( -multivariate_mean(w,mu) - Zcf*StdDev.MM(w,sigma) )
>>> }
>>>
>>> #call revised mVAR.MM with m3 and m4 equal 0
>>> MVaR1 = mVaR.MM1(w,Mean,Stdev, 0,0, .95);
>>> #shoud be equal to GVaR
>>> GVaR==MVaR1
>>> MVaR1
>>>
>>> #with m3 and m4 not zero
>>> MVaR1 = mVaR.MM1(1, Mean, Stdev, Skewness, ExKurtosis, .95);
>>> MVaR1
>>>
>>> MVaR1 = mVaR.MM1(w, Mean, Stdev, Skewness, ExKurtosis, .95);
>>> MVaR1
>>>
>>> This result still looks strange and would appreciate any thoughts,
>>> with
>>> 1 or w weights, I get the same just scaled.  Note this is real
>>> commodity
>>> data with all statistics generated by table.Stats.
>>>
>>> Thanks
>>>
>>> Joe
>>>
>>>
>>> --
>>> *Joe W. Byers*
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R questions
>>> should go.


--
*Joe W. Byers*

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

[Fwd: Re: Fwd: Re: [Fwd: Performance Analytics internal multivariateMoments calculations]]

braverock
Dries is our Google Summer of Code student working on this project for
GSoC 2017:

https://github.com/rstats-gsoc/gsoc2017/wiki/Improved-Functionality-for
-Higher-Order-Comoment-Estimation-in-PerformanceAnalytics

A working paper discussing co-moment estimation that goes into more
detail about the sparse forms is here:

https://ssrn.com/abstract=2839781

with supplemental details here:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2970015

By the end of summer, all the methods in PerformanceAnalytics should be
using the sparse forms, which offer more than an order of magnitude
faster performance, along with some other improvements for numerical
stability of the estimation, and a framework for more estimating
methods to be added over time.

Regards,

Brian

-------- Forwarded Message --------
From: Joe W. Byers via R-SIG-Finance <[hidden email]>
Reply-to: "Joe W. Byers" <[hidden email]>
To: Brian G. Peterson <[hidden email]>, Dries Cornilly <dries.corn
[hidden email]>, [hidden email]
Subject: Re: [R-SIG-Finance] Fwd: Re: [Fwd: Performance Analytics
internal multivariateMoments calculations]
Date: Tue, 23 May 2017 10:50:37 -0400

Gentlemen,

I so appreciate this feedback.  I do have a followup just so I fully 
understand.  The functions mVaR.MM and mES.MM both require the raw 3rd 
and 4th moments, not the standardized skewness and kurtosis
statistics.  
And, would Dries mind sharing the simplified formulas that he mentions.

Again, thank you both very much.


Joe

On 05/23/2017 05:14 AM, Brian G. Peterson wrote:

>
>
>
> -------- Forwarded Message --------
> Subject:     Re: [Fwd: [R-SIG-Finance] Performance Analytics
> internal 
> multivariateMoments calculations]
> Date:     Tue, 23 May 2017 02:10:21 +0000
> From:     Dries Cornilly <[hidden email]>
> To:     Brian G. Peterson <[hidden email]>
>
>
>
> Also, Joe Byers did not run in the same issue. He is confusing the 
> kurtosis and the excess kurtosis.
>
> The function “kurtosis.MM” in “mVaR.MM” does not return the excess 
> kurtosis, but rather m4 / sd^4 and hence the -3 still needs to be 
> done. His example where he claims that GVaR and MVaR should return
> the 
> same is not correct. He uses 0 as fourth order moment in the input, 
> which is only valid for a degenerate distribution. The input for 
> equality should rather be
> GVaR(w, Mean, Stdev, .95)
> MVaR(w, Mean, Stdev, 0, 3 * Stdev^4, .95) # note the difference in 
> fourth moment.
> For a Gaussian the excess kurtosis is zero, or equivalently, the 
> standardised fourth moment (m4 / sd^4) is equal to 3, but the fourth 
> moment of a Gaussian is not equal to zero, it is equal to a function 
> of the variance.
>
> Additionally, he will generate problems when using the skewness
> input 
> as in his ‘corrected’ version. The function “mVaR.MM” takes the raw 
> third order central moment(s) and standardises in the function
> itself, 
> whereas his altered function will standardise again on the already 
> standardised input and hence give a wrong result.
>
> Regards
> Dries
>
>
> On 22 May 2017 at 20:32:04, Dries Cornilly ([hidden email]
> e 
> <mailto:[hidden email]>) wrote:
>
> > I showed it to Joshua this morning and it seemed like a pure
> > integer
> > problem.
> >
> > library(PerformanceAnalytics)
> > X <- matrix(1:12, ncol = 3)
> > M3.MM(X) # this one behaves strangely (at this time, X is still
> > filled
> > with integers)
> > M3.MM(X * 1.0)  # multiplying by 1.0 to cast to double provides a
> > zero
> > coskewness matrix at it should
> >
> > However, I do have some code replicating the modified VaR and
> > modified
> > Expected shortfall using simplified formulas (coming from a working
> > paper of Doug Martin if I recall correctly). I was going to suggest
> > to
> > replace it since the output is identical and it seems more stable
> > to
> > compute. When I have the alternative moment estimators using the
> > unique elements, it might be a good idea to replace the inside of
> > mVaR
> > and mES using the multivariate moments to work with the vector of
> > unique elements (and extract the unique elements if the full
> > matrices
> > are given). This will be a good memory improvement, especially when
> > computing the portfolio moments for mVaR and mES and the
> > derivatives
> > needed for the component VaR and component ES.
> >
> >
> > Regards
> > Dries
> >
> >
> > On 22 May 2017 at 16:20:22, Brian G. Peterson ([hidden email]
> > <mailto:[hidden email]>) wrote:
> >
> > > Looks like Joe Byers ran into the same issue you found earlier
> > > today.
> > >
> > > -- 
> > > Brian G. Peterson
> > > http://braverock.com/brian/
> > > Ph: 773-459-4973
> > > IM: bgpbraverock
> > >
> > > -------- Forwarded Message --------
> > > From: Joe W. Byers via R-SIG-Finance <[hidden email]
> > > >
> > > Reply-to: "Joe W. Byers" <[hidden email]>
> > > To: [hidden email]
> > > Subject: [R-SIG-Finance] Performance Analytics internal
> > > multivariateMoments calculations
> > > Date: Mon, 22 May 2017 15:59:59 -0400
> > >
> > > All,
> > >
> > >
> > > I have carved on all methods in MultivariateMoments.R so I can
> > > all
> > > them
> > > directly
> > >
> > > I am working on the modified var calculations.  I ran the mVaR.MM
> > > on
> > > my
> > > data and the results were odd.  I reran setting skewness and
> > > kurtosis
> > > to
> > > 0 to compare with GVAR.MM.  Still questions.  I have the
> > > following
> > > example
> > >
> > > #test for sigfinance
> > > w = 1000000;
> > > Mean = 0.0001898251;
> > > Stdev = 0.01612464;
> > > ExKurtosis = 3.946156;
> > > Skewness = -0.1373454;
> > >
> > > GVaR = GVaR.MM(w,Mean,Stdev, .95)
> > > GVaR
> > > MVaR = mVaR.MM(w,Mean,Stdev, 0,0, .95);
> > > #shoud be equal to GVaR
> > > GVaR==MVaR
> > > MVaR
> > >
> > > mVaR.MM does not return the GVaR.MM.  I found the exkurt was not
> > > zero
> > > as
> > > I think it should be.  I remove the - 3 from that line and exkurt
> > > became
> > > zero and mVaR.MM == GVaR results.  I have included this modified
> > > version
> > > of mVaR.MM below and continued the test.  Is this an issue or am
> > > I
> > > missing something?
> > >
> > >
> > > #corrected exkurt calc
> > > mVaR.MM1 = function(w, mu, sigma, M3, M4, p ){
> > >    skew = skewness.MM(w,sigma,M3);
> > >    exkurt = kurtosis.MM(w,sigma,M4); #removed -3
> > >    z = qnorm(1-p);
> > >    zc = z + (1/6)*(z^2 -1)*skew
> > >    Zcf = zc + (1/24)*(z^3 - 3*z)*exkurt - (1/36)*(2*z^3 -
> > > 5*z)*skew^2;
> > >    return ( -multivariate_mean(w,mu) - Zcf*StdDev.MM(w,sigma) )
> > > }
> > >
> > > #call revised mVAR.MM with m3 and m4 equal 0
> > > MVaR1 = mVaR.MM1(w,Mean,Stdev, 0,0, .95);
> > > #shoud be equal to GVaR
> > > GVaR==MVaR1
> > > MVaR1
> > >
> > > #with m3 and m4 not zero
> > > MVaR1 = mVaR.MM1(1, Mean, Stdev, Skewness, ExKurtosis, .95);
> > > MVaR1
> > >
> > > MVaR1 = mVaR.MM1(w, Mean, Stdev, Skewness, ExKurtosis, .95);
> > > MVaR1
> > >
> > > This result still looks strange and would appreciate any
> > > thoughts,
> > > with
> > > 1 or w weights, I get the same just scaled.  Note this is real
> > > commodity
> > > data with all statistics generated by table.Stats.
> > >
> > > Thanks
> > >
> > > Joe
> > >
> > >
> > > -- 
> > > *Joe W. Byers*
> > >

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.