Detail analysis of solvers' results

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Detail analysis of solvers' results

GALIB KHAN
Alexios,

I updated my script to use multiple solvers and compare results.

It looks like nlminb is the most accurate solver for this generic dataset
that I am using.

I posted the results on stack exchange so its easier to view the results.

If you want, I can post the results here.

Here is the url with the updated script and results:

https://stackoverflow.com/questions/51900177/should-the-positioning-of-the-external-regressors-change-the-output-of-arma-garc/


On Mon, Aug 20, 2018, 7:29 AM GALIB KHAN <[hidden email]>
wrote:

> Alexios,
>
> I changed the solver to "hybrid" but kept the scaling that you provided
> and got different results when switching the positions for the external
> regressors.
>
> ugarchfit(spec = spec, data = as.matrix(temp$y),solver =
> "hybrid",fit.control=list(scale=1))
>
> So it looks like you have to change the solver to "nlminb" and update
> fit.control with the scaling that you provided.
>
> Galib
>
>
> On Sun, Aug 19, 2018 at 11:41 PM, GALIB KHAN <
> [hidden email]> wrote:
>
>> Alexios,
>>
>>
>> Veryyyy interesting!!!!! No I cannot see any differences at all lol.
>>
>> I updated the code and indeed you are correct sir. Thank you for your
>> time in investigating this.
>>
>> I will update my stack exchange post to reflect your answer in the
>> morning.
>>
>> Again thank you for all your help!!!!
>>
>> Best,
>> Galib Khan
>>
>>
>> On Sun, Aug 19, 2018 at 11:19 PM, alexios galanos <[hidden email]>
>> wrote:
>>
>>> I did use the seed you provided.
>>>
>>> Use the following code for estimation:
>>>
>>> fit <- ugarchfit(spec = spec, data = as.matrix(temp$y),solver =
>>> "nlminb", fit.control=list(scale=1))
>>>
>>> model_maker(var1)
>>>          Estimate  Std. Error     t value     Pr(>|t|)
>>> mu     -7.3998577  0.69086641 -10.7109821 0.0000000000
>>> ar1     0.3387323  0.08280162   4.0908900 0.0000429721
>>> ar2    -0.8834201  0.06569477 -13.4473414 0.0000000000
>>> ma1    -0.2902069  0.08598589  -3.3750525 0.0007380161
>>> ma2     0.8660807  0.06778418  12.7770320 0.0000000000
>>> mxreg1  1.6782992  0.12769644  13.1428825 0.0000000000
>>> mxreg2  2.5225382  0.04292728  58.7630625 0.0000000000
>>> omega  12.0047145  0.82986864  14.4658010 0.0000000000
>>> alpha1  0.0000000  0.07358520   0.0000000 1.0000000000
>>> shape  63.0103309 98.49188643   0.6397515 0.5223341761
>>>
>>> model_maker(var2)
>>>          Estimate  Std. Error     t value     Pr(>|t|)
>>> mu     -7.3998549  0.69086651 -10.7109764 0.000000e+00
>>> ar1     0.3387334  0.08280150   4.0909088 4.296861e-05
>>> ar2    -0.8834206  0.06569433 -13.4474406 0.000000e+00
>>> ma1    -0.2902081  0.08598562  -3.3750776 7.379487e-04
>>> ma2     0.8660811  0.06778412  12.7770487 0.000000e+00
>>> mxreg1  2.5225383  0.04292728  58.7630642 0.000000e+00
>>> mxreg2  1.6782987  0.12769640  13.1428817 0.000000e+00
>>> omega  12.0047142  0.82992363  14.4648419 0.000000e+00
>>> alpha1  0.0000000  0.07359329   0.0000000 1.000000e+00
>>> shape  63.0105962 98.49368444   0.6397425 5.223400e-01
>>>
>>>
>>> I can’t see any “significant” differences, can you?
>>> It’s completely related to the optimization/starting parameters. The
>>> “scale” is documented and not on by default (perhaps it should be).
>>>
>>> Alexios
>>>
>>>
>>> > On Aug 19, 2018, at 9:02 PM, GALIB KHAN <[hidden email]>
>>> wrote:
>>> >
>>> > Sorry for sending this again, I didn't include r-sig-finance in the
>>> email address. I'm still adjusting in how to respond.
>>> >
>>> > Alexios,
>>> >
>>> > Did you set the set the seed to 1, because I'm looking at your results
>>> and the numbers do not match with the numbers that I have provided.
>>> >
>>> > I understand why the coefficients' estimates are similar but it
>>> doesn't explain why other columns such as the t-value and pr are off by a
>>> large margin. Also estimates for mu, ar*, ma*, omega, alpha1, and shape may
>>> have large differences.
>>> >
>>> > Take mu as an example:
>>> > -7.538187e+00 - (-7.877120e+00) = 0.338933, isn't that considered a
>>> large difference to the point where it's safe to say that these two values
>>> are not similar?
>>> >
>>> > Another example is the t-values for x1 and x2:
>>> > x1 = 8.799994e+01   -  5.509361e+02 = -462.9362
>>> > x2 = 8.508606e+01   -  5.287634e+02 = -443.6773
>>> >
>>> > An more alarming case that unfortunately I cannot share due to the
>>> data being sensitive is that when the x variables' positions are switched,
>>> the p-values are not the same. The p-value for a particular external
>>> regressor went from 0 to 0.4385.
>>> >
>>> > I will attempt to re-create a separate generic dataset that is similar
>>> to the sensitive data that I am using.
>>> >
>>> >
>>> > Galib Khan
>>> >
>>> >
>>> > On Sun, Aug 19, 2018 at 10:06 PM, alexios galanos <[hidden email]>
>>> wrote:
>>> > I run the code you provided and obtain the following results related
>>> to the external parameters:
>>> >
>>> >
>>> > Case 1 (x1,x2)
>>> > # x2 is second
>>> >
>>> >             Estimate   Std. Error       t value  Pr(>|t|)
>>> > mxreg1  1.6724148 1.203377e-01  1.389767e+01 0.0000000
>>> > mxreg2  2.5310286 1.878833e-02  1.347128e+02 0.0000000
>>> >
>>> > Case 2 (x2,x1)
>>> > # i.e. x2 is now first
>>> >
>>> > mxreg1  2.5225382  0.04292725  58.7631024 0.000000e+00
>>> > mxreg2  1.6782986  0.12769622  13.1428990 0.000000e+00
>>> >
>>> > Small differences in the coefficients are the result of the optimizer.
>>> There may be an issues in the
>>> > way starting parameters are being generated based on some recent input
>>> from Josh Ulrich (still to investigate)
>>> > and related to arima0 (used to generate start parameters), but
>>> otherwise don’t see a large problem at first glance.
>>> >
>>> > Alexios
>>> >
>>> > > On Aug 19, 2018, at 5:46 PM, GALIB KHAN <
>>> [hidden email]> wrote:
>>> > >
>>> > > Recently I have discovered a problem with a package called rugarch
>>> that
>>> > > creates arma-garch models. The issue is that if you literally change
>>> the
>>> > > positions of the x variables (external regressors) then you get two
>>> > > completely different results.
>>> > >
>>> > > In other words:
>>> > >
>>> > >   - model1 = (arma(2,2) + garch(1,0) + x1 + x2)
>>> > >   - model2 = (arma(2,2) + garch(1,0) + x2 + x1)
>>> > >   - rugarch's output is essentially saying that model1 != model2
>>> > >   - When the correct result should be model1 == model2
>>> > >
>>> > > I may not know a lot of statistics but I know for a fact that if you
>>> move
>>> > > the x variables around, the output should still be the same.
>>> > >
>>> > > Am I wrong on this?
>>> > >
>>> > > Here's my stack exchange post that shows a generic R script proving
>>> my
>>> > > point: Should the positioning of the external regressors change the
>>> output
>>> > > of arma-garch? (Possible rugarch bug/error)
>>> > > <
>>> https://stackoverflow.com/questions/51900177/should-the-positioning-of-the-external-regressors-change-the-output-of-arma-garc
>>> >
>>> > >
>>> > > Any feedback is welcomed.
>>> > >
>>> > > Thanks
>>> > >
>>> > >       [[alternative HTML version deleted]]
>>> > >
>>> > > _______________________________________________
>>> > > [hidden email] mailing list
>>> > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> > > -- Subscriber-posting only. If you want to post, subscribe first.
>>> > > -- Also note that this is not the r-help list where general R
>>> questions should go.
>>> > >
>>> >
>>> >
>>>
>>>
>>
>
> Galib Khan
>
>
> On Sun, Aug 19, 2018 at 11:41 PM, GALIB KHAN <
> [hidden email]> wrote:
>
>> Alexios,
>>
>>
>> Veryyyy interesting!!!!! No I cannot see any differences at all lol.
>>
>> I updated the code and indeed you are correct sir. Thank you for your
>> time in investigating this.
>>
>> I will update my stack exchange post to reflect your answer in the
>> morning.
>>
>> Again thank you for all your help!!!!
>>
>> Best,
>> Galib Khan
>>
>>
>> On Sun, Aug 19, 2018 at 11:19 PM, alexios galanos <[hidden email]>
>> wrote:
>>
>>> I did use the seed you provided.
>>>
>>> Use the following code for estimation:
>>>
>>> fit <- ugarchfit(spec = spec, data = as.matrix(temp$y),solver =
>>> "nlminb", fit.control=list(scale=1))
>>>
>>> model_maker(var1)
>>>          Estimate  Std. Error     t value     Pr(>|t|)
>>> mu     -7.3998577  0.69086641 -10.7109821 0.0000000000
>>> ar1     0.3387323  0.08280162   4.0908900 0.0000429721
>>> ar2    -0.8834201  0.06569477 -13.4473414 0.0000000000
>>> ma1    -0.2902069  0.08598589  -3.3750525 0.0007380161
>>> ma2     0.8660807  0.06778418  12.7770320 0.0000000000
>>> mxreg1  1.6782992  0.12769644  13.1428825 0.0000000000
>>> mxreg2  2.5225382  0.04292728  58.7630625 0.0000000000
>>> omega  12.0047145  0.82986864  14.4658010 0.0000000000
>>> alpha1  0.0000000  0.07358520   0.0000000 1.0000000000
>>> shape  63.0103309 98.49188643   0.6397515 0.5223341761
>>>
>>> model_maker(var2)
>>>          Estimate  Std. Error     t value     Pr(>|t|)
>>> mu     -7.3998549  0.69086651 -10.7109764 0.000000e+00
>>> ar1     0.3387334  0.08280150   4.0909088 4.296861e-05
>>> ar2    -0.8834206  0.06569433 -13.4474406 0.000000e+00
>>> ma1    -0.2902081  0.08598562  -3.3750776 7.379487e-04
>>> ma2     0.8660811  0.06778412  12.7770487 0.000000e+00
>>> mxreg1  2.5225383  0.04292728  58.7630642 0.000000e+00
>>> mxreg2  1.6782987  0.12769640  13.1428817 0.000000e+00
>>> omega  12.0047142  0.82992363  14.4648419 0.000000e+00
>>> alpha1  0.0000000  0.07359329   0.0000000 1.000000e+00
>>> shape  63.0105962 98.49368444   0.6397425 5.223400e-01
>>>
>>>
>>> I can’t see any “significant” differences, can you?
>>> It’s completely related to the optimization/starting parameters. The
>>> “scale” is documented and not on by default (perhaps it should be).
>>>
>>> Alexios
>>>
>>>
>>> > On Aug 19, 2018, at 9:02 PM, GALIB KHAN <[hidden email]>
>>> wrote:
>>> >
>>> > Sorry for sending this again, I didn't include r-sig-finance in the
>>> email address. I'm still adjusting in how to respond.
>>> >
>>> > Alexios,
>>> >
>>> > Did you set the set the seed to 1, because I'm looking at your results
>>> and the numbers do not match with the numbers that I have provided.
>>> >
>>> > I understand why the coefficients' estimates are similar but it
>>> doesn't explain why other columns such as the t-value and pr are off by a
>>> large margin. Also estimates for mu, ar*, ma*, omega, alpha1, and shape may
>>> have large differences.
>>> >
>>> > Take mu as an example:
>>> > -7.538187e+00 - (-7.877120e+00) = 0.338933, isn't that considered a
>>> large difference to the point where it's safe to say that these two values
>>> are not similar?
>>> >
>>> > Another example is the t-values for x1 and x2:
>>> > x1 = 8.799994e+01   -  5.509361e+02 = -462.9362
>>> > x2 = 8.508606e+01   -  5.287634e+02 = -443.6773
>>> >
>>> > An more alarming case that unfortunately I cannot share due to the
>>> data being sensitive is that when the x variables' positions are switched,
>>> the p-values are not the same. The p-value for a particular external
>>> regressor went from 0 to 0.4385.
>>> >
>>> > I will attempt to re-create a separate generic dataset that is similar
>>> to the sensitive data that I am using.
>>> >
>>> >
>>> > Galib Khan
>>> >
>>> >
>>> > On Sun, Aug 19, 2018 at 10:06 PM, alexios galanos <[hidden email]>
>>> wrote:
>>> > I run the code you provided and obtain the following results related
>>> to the external parameters:
>>> >
>>> >
>>> > Case 1 (x1,x2)
>>> > # x2 is second
>>> >
>>> >             Estimate   Std. Error       t value  Pr(>|t|)
>>> > mxreg1  1.6724148 1.203377e-01  1.389767e+01 0.0000000
>>> > mxreg2  2.5310286 1.878833e-02  1.347128e+02 0.0000000
>>> >
>>> > Case 2 (x2,x1)
>>> > # i.e. x2 is now first
>>> >
>>> > mxreg1  2.5225382  0.04292725  58.7631024 0.000000e+00
>>> > mxreg2  1.6782986  0.12769622  13.1428990 0.000000e+00
>>> >
>>> > Small differences in the coefficients are the result of the optimizer.
>>> There may be an issues in the
>>> > way starting parameters are being generated based on some recent input
>>> from Josh Ulrich (still to investigate)
>>> > and related to arima0 (used to generate start parameters), but
>>> otherwise don’t see a large problem at first glance.
>>> >
>>> > Alexios
>>> >
>>> > > On Aug 19, 2018, at 5:46 PM, GALIB KHAN <
>>> [hidden email]> wrote:
>>> > >
>>> > > Recently I have discovered a problem with a package called rugarch
>>> that
>>> > > creates arma-garch models. The issue is that if you literally change
>>> the
>>> > > positions of the x variables (external regressors) then you get two
>>> > > completely different results.
>>> > >
>>> > > In other words:
>>> > >
>>> > >   - model1 = (arma(2,2) + garch(1,0) + x1 + x2)
>>> > >   - model2 = (arma(2,2) + garch(1,0) + x2 + x1)
>>> > >   - rugarch's output is essentially saying that model1 != model2
>>> > >   - When the correct result should be model1 == model2
>>> > >
>>> > > I may not know a lot of statistics but I know for a fact that if you
>>> move
>>> > > the x variables around, the output should still be the same.
>>> > >
>>> > > Am I wrong on this?
>>> > >
>>> > > Here's my stack exchange post that shows a generic R script proving
>>> my
>>> > > point: Should the positioning of the external regressors change the
>>> output
>>> > > of arma-garch? (Possible rugarch bug/error)
>>> > > <
>>> https://stackoverflow.com/questions/51900177/should-the-positioning-of-the-external-regressors-change-the-output-of-arma-garc
>>> >
>>> > >
>>> > > Any feedback is welcomed.
>>> > >
>>> > > Thanks
>>> > >
>>> > >       [[alternative HTML version deleted]]
>>> > >
>>> > > _______________________________________________
>>> > > [hidden email] mailing list
>>> > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> > > -- Subscriber-posting only. If you want to post, subscribe first.
>>> > > -- Also note that this is not the r-help list where general R
>>> questions should go.
>>> > >
>>> >
>>> >
>>>
>>>
>>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Detail analysis of solvers' results

GALIB KHAN
This post is an accident. Please ignore it.

On Mon, Aug 20, 2018, 3:29 PM GALIB KHAN <[hidden email]>
wrote:

> Alexios,
>
> I updated my script to use multiple solvers and compare results.
>
> It looks like nlminb is the most accurate solver for this generic dataset
> that I am using.
>
> I posted the results on stack exchange so its easier to view the results.
>
> If you want, I can post the results here.
>
> Here is the url with the updated script and results:
>
>
> https://stackoverflow.com/questions/51900177/should-the-positioning-of-the-external-regressors-change-the-output-of-arma-garc/
>
>
> On Mon, Aug 20, 2018, 7:29 AM GALIB KHAN <[hidden email]>
> wrote:
>
>> Alexios,
>>
>> I changed the solver to "hybrid" but kept the scaling that you provided
>> and got different results when switching the positions for the external
>> regressors.
>>
>> ugarchfit(spec = spec, data = as.matrix(temp$y),solver =
>> "hybrid",fit.control=list(scale=1))
>>
>> So it looks like you have to change the solver to "nlminb" and update
>> fit.control with the scaling that you provided.
>>
>> Galib
>>
>>
>> On Sun, Aug 19, 2018 at 11:41 PM, GALIB KHAN <
>> [hidden email]> wrote:
>>
>>> Alexios,
>>>
>>>
>>> Veryyyy interesting!!!!! No I cannot see any differences at all lol.
>>>
>>> I updated the code and indeed you are correct sir. Thank you for your
>>> time in investigating this.
>>>
>>> I will update my stack exchange post to reflect your answer in the
>>> morning.
>>>
>>> Again thank you for all your help!!!!
>>>
>>> Best,
>>> Galib Khan
>>>
>>>
>>> On Sun, Aug 19, 2018 at 11:19 PM, alexios galanos <[hidden email]>
>>> wrote:
>>>
>>>> I did use the seed you provided.
>>>>
>>>> Use the following code for estimation:
>>>>
>>>> fit <- ugarchfit(spec = spec, data = as.matrix(temp$y),solver =
>>>> "nlminb", fit.control=list(scale=1))
>>>>
>>>> model_maker(var1)
>>>>          Estimate  Std. Error     t value     Pr(>|t|)
>>>> mu     -7.3998577  0.69086641 -10.7109821 0.0000000000
>>>> ar1     0.3387323  0.08280162   4.0908900 0.0000429721
>>>> ar2    -0.8834201  0.06569477 -13.4473414 0.0000000000
>>>> ma1    -0.2902069  0.08598589  -3.3750525 0.0007380161
>>>> ma2     0.8660807  0.06778418  12.7770320 0.0000000000
>>>> mxreg1  1.6782992  0.12769644  13.1428825 0.0000000000
>>>> mxreg2  2.5225382  0.04292728  58.7630625 0.0000000000
>>>> omega  12.0047145  0.82986864  14.4658010 0.0000000000
>>>> alpha1  0.0000000  0.07358520   0.0000000 1.0000000000
>>>> shape  63.0103309 98.49188643   0.6397515 0.5223341761
>>>>
>>>> model_maker(var2)
>>>>          Estimate  Std. Error     t value     Pr(>|t|)
>>>> mu     -7.3998549  0.69086651 -10.7109764 0.000000e+00
>>>> ar1     0.3387334  0.08280150   4.0909088 4.296861e-05
>>>> ar2    -0.8834206  0.06569433 -13.4474406 0.000000e+00
>>>> ma1    -0.2902081  0.08598562  -3.3750776 7.379487e-04
>>>> ma2     0.8660811  0.06778412  12.7770487 0.000000e+00
>>>> mxreg1  2.5225383  0.04292728  58.7630642 0.000000e+00
>>>> mxreg2  1.6782987  0.12769640  13.1428817 0.000000e+00
>>>> omega  12.0047142  0.82992363  14.4648419 0.000000e+00
>>>> alpha1  0.0000000  0.07359329   0.0000000 1.000000e+00
>>>> shape  63.0105962 98.49368444   0.6397425 5.223400e-01
>>>>
>>>>
>>>> I can’t see any “significant” differences, can you?
>>>> It’s completely related to the optimization/starting parameters. The
>>>> “scale” is documented and not on by default (perhaps it should be).
>>>>
>>>> Alexios
>>>>
>>>>
>>>> > On Aug 19, 2018, at 9:02 PM, GALIB KHAN <
>>>> [hidden email]> wrote:
>>>> >
>>>> > Sorry for sending this again, I didn't include r-sig-finance in the
>>>> email address. I'm still adjusting in how to respond.
>>>> >
>>>> > Alexios,
>>>> >
>>>> > Did you set the set the seed to 1, because I'm looking at your
>>>> results and the numbers do not match with the numbers that I have provided.
>>>> >
>>>> > I understand why the coefficients' estimates are similar but it
>>>> doesn't explain why other columns such as the t-value and pr are off by a
>>>> large margin. Also estimates for mu, ar*, ma*, omega, alpha1, and shape may
>>>> have large differences.
>>>> >
>>>> > Take mu as an example:
>>>> > -7.538187e+00 - (-7.877120e+00) = 0.338933, isn't that considered a
>>>> large difference to the point where it's safe to say that these two values
>>>> are not similar?
>>>> >
>>>> > Another example is the t-values for x1 and x2:
>>>> > x1 = 8.799994e+01   -  5.509361e+02 = -462.9362
>>>> > x2 = 8.508606e+01   -  5.287634e+02 = -443.6773
>>>> >
>>>> > An more alarming case that unfortunately I cannot share due to the
>>>> data being sensitive is that when the x variables' positions are switched,
>>>> the p-values are not the same. The p-value for a particular external
>>>> regressor went from 0 to 0.4385.
>>>> >
>>>> > I will attempt to re-create a separate generic dataset that is
>>>> similar to the sensitive data that I am using.
>>>> >
>>>> >
>>>> > Galib Khan
>>>> >
>>>> >
>>>> > On Sun, Aug 19, 2018 at 10:06 PM, alexios galanos <
>>>> [hidden email]> wrote:
>>>> > I run the code you provided and obtain the following results related
>>>> to the external parameters:
>>>> >
>>>> >
>>>> > Case 1 (x1,x2)
>>>> > # x2 is second
>>>> >
>>>> >             Estimate   Std. Error       t value  Pr(>|t|)
>>>> > mxreg1  1.6724148 1.203377e-01  1.389767e+01 0.0000000
>>>> > mxreg2  2.5310286 1.878833e-02  1.347128e+02 0.0000000
>>>> >
>>>> > Case 2 (x2,x1)
>>>> > # i.e. x2 is now first
>>>> >
>>>> > mxreg1  2.5225382  0.04292725  58.7631024 0.000000e+00
>>>> > mxreg2  1.6782986  0.12769622  13.1428990 0.000000e+00
>>>> >
>>>> > Small differences in the coefficients are the result of the
>>>> optimizer. There may be an issues in the
>>>> > way starting parameters are being generated based on some recent
>>>> input from Josh Ulrich (still to investigate)
>>>> > and related to arima0 (used to generate start parameters), but
>>>> otherwise don’t see a large problem at first glance.
>>>> >
>>>> > Alexios
>>>> >
>>>> > > On Aug 19, 2018, at 5:46 PM, GALIB KHAN <
>>>> [hidden email]> wrote:
>>>> > >
>>>> > > Recently I have discovered a problem with a package called rugarch
>>>> that
>>>> > > creates arma-garch models. The issue is that if you literally
>>>> change the
>>>> > > positions of the x variables (external regressors) then you get two
>>>> > > completely different results.
>>>> > >
>>>> > > In other words:
>>>> > >
>>>> > >   - model1 = (arma(2,2) + garch(1,0) + x1 + x2)
>>>> > >   - model2 = (arma(2,2) + garch(1,0) + x2 + x1)
>>>> > >   - rugarch's output is essentially saying that model1 != model2
>>>> > >   - When the correct result should be model1 == model2
>>>> > >
>>>> > > I may not know a lot of statistics but I know for a fact that if
>>>> you move
>>>> > > the x variables around, the output should still be the same.
>>>> > >
>>>> > > Am I wrong on this?
>>>> > >
>>>> > > Here's my stack exchange post that shows a generic R script proving
>>>> my
>>>> > > point: Should the positioning of the external regressors change the
>>>> output
>>>> > > of arma-garch? (Possible rugarch bug/error)
>>>> > > <
>>>> https://stackoverflow.com/questions/51900177/should-the-positioning-of-the-external-regressors-change-the-output-of-arma-garc
>>>> >
>>>> > >
>>>> > > Any feedback is welcomed.
>>>> > >
>>>> > > Thanks
>>>> > >
>>>> > >       [[alternative HTML version deleted]]
>>>> > >
>>>> > > _______________________________________________
>>>> > > [hidden email] mailing list
>>>> > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>> > > -- Subscriber-posting only. If you want to post, subscribe first.
>>>> > > -- Also note that this is not the r-help list where general R
>>>> questions should go.
>>>> > >
>>>> >
>>>> >
>>>>
>>>>
>>>
>>
>> Galib Khan
>>
>>
>> On Sun, Aug 19, 2018 at 11:41 PM, GALIB KHAN <
>> [hidden email]> wrote:
>>
>>> Alexios,
>>>
>>>
>>> Veryyyy interesting!!!!! No I cannot see any differences at all lol.
>>>
>>> I updated the code and indeed you are correct sir. Thank you for your
>>> time in investigating this.
>>>
>>> I will update my stack exchange post to reflect your answer in the
>>> morning.
>>>
>>> Again thank you for all your help!!!!
>>>
>>> Best,
>>> Galib Khan
>>>
>>>
>>> On Sun, Aug 19, 2018 at 11:19 PM, alexios galanos <[hidden email]>
>>> wrote:
>>>
>>>> I did use the seed you provided.
>>>>
>>>> Use the following code for estimation:
>>>>
>>>> fit <- ugarchfit(spec = spec, data = as.matrix(temp$y),solver =
>>>> "nlminb", fit.control=list(scale=1))
>>>>
>>>> model_maker(var1)
>>>>          Estimate  Std. Error     t value     Pr(>|t|)
>>>> mu     -7.3998577  0.69086641 -10.7109821 0.0000000000
>>>> ar1     0.3387323  0.08280162   4.0908900 0.0000429721
>>>> ar2    -0.8834201  0.06569477 -13.4473414 0.0000000000
>>>> ma1    -0.2902069  0.08598589  -3.3750525 0.0007380161
>>>> ma2     0.8660807  0.06778418  12.7770320 0.0000000000
>>>> mxreg1  1.6782992  0.12769644  13.1428825 0.0000000000
>>>> mxreg2  2.5225382  0.04292728  58.7630625 0.0000000000
>>>> omega  12.0047145  0.82986864  14.4658010 0.0000000000
>>>> alpha1  0.0000000  0.07358520   0.0000000 1.0000000000
>>>> shape  63.0103309 98.49188643   0.6397515 0.5223341761
>>>>
>>>> model_maker(var2)
>>>>          Estimate  Std. Error     t value     Pr(>|t|)
>>>> mu     -7.3998549  0.69086651 -10.7109764 0.000000e+00
>>>> ar1     0.3387334  0.08280150   4.0909088 4.296861e-05
>>>> ar2    -0.8834206  0.06569433 -13.4474406 0.000000e+00
>>>> ma1    -0.2902081  0.08598562  -3.3750776 7.379487e-04
>>>> ma2     0.8660811  0.06778412  12.7770487 0.000000e+00
>>>> mxreg1  2.5225383  0.04292728  58.7630642 0.000000e+00
>>>> mxreg2  1.6782987  0.12769640  13.1428817 0.000000e+00
>>>> omega  12.0047142  0.82992363  14.4648419 0.000000e+00
>>>> alpha1  0.0000000  0.07359329   0.0000000 1.000000e+00
>>>> shape  63.0105962 98.49368444   0.6397425 5.223400e-01
>>>>
>>>>
>>>> I can’t see any “significant” differences, can you?
>>>> It’s completely related to the optimization/starting parameters. The
>>>> “scale” is documented and not on by default (perhaps it should be).
>>>>
>>>> Alexios
>>>>
>>>>
>>>> > On Aug 19, 2018, at 9:02 PM, GALIB KHAN <
>>>> [hidden email]> wrote:
>>>> >
>>>> > Sorry for sending this again, I didn't include r-sig-finance in the
>>>> email address. I'm still adjusting in how to respond.
>>>> >
>>>> > Alexios,
>>>> >
>>>> > Did you set the set the seed to 1, because I'm looking at your
>>>> results and the numbers do not match with the numbers that I have provided.
>>>> >
>>>> > I understand why the coefficients' estimates are similar but it
>>>> doesn't explain why other columns such as the t-value and pr are off by a
>>>> large margin. Also estimates for mu, ar*, ma*, omega, alpha1, and shape may
>>>> have large differences.
>>>> >
>>>> > Take mu as an example:
>>>> > -7.538187e+00 - (-7.877120e+00) = 0.338933, isn't that considered a
>>>> large difference to the point where it's safe to say that these two values
>>>> are not similar?
>>>> >
>>>> > Another example is the t-values for x1 and x2:
>>>> > x1 = 8.799994e+01   -  5.509361e+02 = -462.9362
>>>> > x2 = 8.508606e+01   -  5.287634e+02 = -443.6773
>>>> >
>>>> > An more alarming case that unfortunately I cannot share due to the
>>>> data being sensitive is that when the x variables' positions are switched,
>>>> the p-values are not the same. The p-value for a particular external
>>>> regressor went from 0 to 0.4385.
>>>> >
>>>> > I will attempt to re-create a separate generic dataset that is
>>>> similar to the sensitive data that I am using.
>>>> >
>>>> >
>>>> > Galib Khan
>>>> >
>>>> >
>>>> > On Sun, Aug 19, 2018 at 10:06 PM, alexios galanos <
>>>> [hidden email]> wrote:
>>>> > I run the code you provided and obtain the following results related
>>>> to the external parameters:
>>>> >
>>>> >
>>>> > Case 1 (x1,x2)
>>>> > # x2 is second
>>>> >
>>>> >             Estimate   Std. Error       t value  Pr(>|t|)
>>>> > mxreg1  1.6724148 1.203377e-01  1.389767e+01 0.0000000
>>>> > mxreg2  2.5310286 1.878833e-02  1.347128e+02 0.0000000
>>>> >
>>>> > Case 2 (x2,x1)
>>>> > # i.e. x2 is now first
>>>> >
>>>> > mxreg1  2.5225382  0.04292725  58.7631024 0.000000e+00
>>>> > mxreg2  1.6782986  0.12769622  13.1428990 0.000000e+00
>>>> >
>>>> > Small differences in the coefficients are the result of the
>>>> optimizer. There may be an issues in the
>>>> > way starting parameters are being generated based on some recent
>>>> input from Josh Ulrich (still to investigate)
>>>> > and related to arima0 (used to generate start parameters), but
>>>> otherwise don’t see a large problem at first glance.
>>>> >
>>>> > Alexios
>>>> >
>>>> > > On Aug 19, 2018, at 5:46 PM, GALIB KHAN <
>>>> [hidden email]> wrote:
>>>> > >
>>>> > > Recently I have discovered a problem with a package called rugarch
>>>> that
>>>> > > creates arma-garch models. The issue is that if you literally
>>>> change the
>>>> > > positions of the x variables (external regressors) then you get two
>>>> > > completely different results.
>>>> > >
>>>> > > In other words:
>>>> > >
>>>> > >   - model1 = (arma(2,2) + garch(1,0) + x1 + x2)
>>>> > >   - model2 = (arma(2,2) + garch(1,0) + x2 + x1)
>>>> > >   - rugarch's output is essentially saying that model1 != model2
>>>> > >   - When the correct result should be model1 == model2
>>>> > >
>>>> > > I may not know a lot of statistics but I know for a fact that if
>>>> you move
>>>> > > the x variables around, the output should still be the same.
>>>> > >
>>>> > > Am I wrong on this?
>>>> > >
>>>> > > Here's my stack exchange post that shows a generic R script proving
>>>> my
>>>> > > point: Should the positioning of the external regressors change the
>>>> output
>>>> > > of arma-garch? (Possible rugarch bug/error)
>>>> > > <
>>>> https://stackoverflow.com/questions/51900177/should-the-positioning-of-the-external-regressors-change-the-output-of-arma-garc
>>>> >
>>>> > >
>>>> > > Any feedback is welcomed.
>>>> > >
>>>> > > Thanks
>>>> > >
>>>> > >       [[alternative HTML version deleted]]
>>>> > >
>>>> > > _______________________________________________
>>>> > > [hidden email] mailing list
>>>> > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>> > > -- Subscriber-posting only. If you want to post, subscribe first.
>>>> > > -- Also note that this is not the r-help list where general R
>>>> questions should go.
>>>> > >
>>>> >
>>>> >
>>>>
>>>>
>>>
>>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.