Apparent bug in rmgarch

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Apparent bug in rmgarch

Josh Segal
Hi Alexios (and others),

First, thank you for creating rmgarch, it's great.

I'm trying to generate simulations using dccsim from a spec, and find that
when I specify a single value for rseed, the simulation results are not
reproducible (that is, each run gives different results).  I can avoid this
issue by either using a fit instead of a spec or by specifying a vector of
m.sim rseeds.

I did some digging in the code, and found in rdcc-main.R:
For fit, line 1224: rseed = c(rseed, (1:m.sim)*(rseed+1))
For spec, line 1493: rseed = c(rseed, as.integer(runif(m.sim, 1,
Sys.time())))

I think this explains my results.  Seems like the latter is incorrect, and
should match the former, no?

Thanks very much,
Josh

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Apparent bug in rmgarch

alexios
Hi Josh,

Thanks for reporting this...I'll investigate further when time permits.

In the meantime, if you let the rseed=NULL, the returned object will
include the seeds used which can then be passed back into the simulation
function to obtain reproducible results.

Here is an example:
###############################
library(rmgarch)
data(dji30retw)
Dat = dji30retw[, 1:3, drop = FALSE]
cnames = colnames(Dat)
uspec = ugarchspec(mean.model = list(armaOrder = c(2,1)), variance.model
= list(garchOrder = c(1,1), model = "eGARCH"), distribution.model = "norm")
spec1 = dccspec(uspec = multispec( replicate(3, uspec) ), dccOrder =
c(1,1), distribution = "mvnorm")
fit1 = dccfit(spec1, data = Dat, fit.control = list(eval.se=FALSE))
uspec = ugarchspec(mean.model = list(armaOrder = c(2,1)), variance.model
= list(garchOrder = c(1,1), model = "eGARCH"), distribution.model = "norm")
vspec = vector(mode = "list", length = 3)
midx = fit1@model$midx
mpars = fit1@model$mpars
# nasty, I know...will try to write some wrapper to perform this
automatically in future.
for(i in 1:3){
   vspec[[i]] = uspec
   setfixed(vspec[[i]])<-as.list(mpars[midx[,i]==1, i])
}
dccfix = as.list(coef(fit1, "dcc"))
spec2 = dccspec(uspec = multispec( vspec ), dccOrder = c(1,1),
distribution = "mvnorm", fixed.pars = dccfix)
# Fitted object
sim1 = dccsim(fitORspec = fit1, n.sim = 1000, n.start = 100, m.sim = 2,
startMethod = "unconditional",
               rseed = NULL)
print(sim1@msim$rseed)
 > 497782424 497782425 995564850
sim2 = dccsim(fitORspec = fit1, n.sim = 1000, n.start = 100, m.sim = 2,
startMethod = "unconditional",
               rseed=sim1@msim$rseed)

all.equal(sim1@msim$simX[[1]],sim2@msim$simX[[1]])
 > TRUE
all.equal(sim1@msim$simX[[2]],sim2@msim$simX[[2]])
 > TRUE
# Specification Object
sims1 = dccsim(fitORspec = spec2, n.sim = 1000, n.start = 100, m.sim =
2, startMethod = "unconditional",
               rseed = NULL, mexsimdata = NULL, vexsimdata = NULL, preQ
= last(rcor(fit1, type = "Q"))[,,1],
               Qbar = fit1@mfit$Qbar)
sims2 = dccsim(fitORspec = spec2, n.sim = 1000, n.start = 100, m.sim =
2, startMethod = "unconditional",
               rseed=sims1@msim$rseed, preQ = last(rcor(fit1, type =
"Q"))[,,1],
               Qbar = fit1@mfit$Qbar)

all.equal(sims1@msim$simX[[1]],sims2@msim$simX[[1]])
 > TRUE
all.equal(sims1@msim$simX[[2]],sims2@msim$simX[[2]])
 > TRUE
###############################

Regards,

Alexios

On 1/23/18 11:34 AM, Josh Segal wrote:

> Hi Alexios (and others),
>
> First, thank you for creating rmgarch, it's great.
>
> I'm trying to generate simulations using dccsim from a spec, and find that
> when I specify a single value for rseed, the simulation results are not
> reproducible (that is, each run gives different results).  I can avoid this
> issue by either using a fit instead of a spec or by specifying a vector of
> m.sim rseeds.
>
> I did some digging in the code, and found in rdcc-main.R:
> For fit, line 1224: rseed = c(rseed, (1:m.sim)*(rseed+1))
> For spec, line 1493: rseed = c(rseed, as.integer(runif(m.sim, 1,
> Sys.time())))
>
> I think this explains my results.  Seems like the latter is incorrect, and
> should match the former, no?
>
> Thanks very much,
> Josh
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Apparent bug in rmgarch

Josh Segal
I'll try that, thanks!

On Wed, Jan 24, 2018 at 10:15 PM, alexios galanos <[hidden email]>
wrote:

> Hi Josh,
>
> Thanks for reporting this...I'll investigate further when time permits.
>
> In the meantime, if you let the rseed=NULL, the returned object will
> include the seeds used which can then be passed back into the simulation
> function to obtain reproducible results.
>
> Here is an example:
> ###############################
> library(rmgarch)
> data(dji30retw)
> Dat = dji30retw[, 1:3, drop = FALSE]
> cnames = colnames(Dat)
> uspec = ugarchspec(mean.model = list(armaOrder = c(2,1)), variance.model =
> list(garchOrder = c(1,1), model = "eGARCH"), distribution.model = "norm")
> spec1 = dccspec(uspec = multispec( replicate(3, uspec) ), dccOrder =
> c(1,1), distribution = "mvnorm")
> fit1 = dccfit(spec1, data = Dat, fit.control = list(eval.se=FALSE))
> uspec = ugarchspec(mean.model = list(armaOrder = c(2,1)), variance.model =
> list(garchOrder = c(1,1), model = "eGARCH"), distribution.model = "norm")
> vspec = vector(mode = "list", length = 3)
> midx = fit1@model$midx
> mpars = fit1@model$mpars
> # nasty, I know...will try to write some wrapper to perform this
> automatically in future.
> for(i in 1:3){
>   vspec[[i]] = uspec
>   setfixed(vspec[[i]])<-as.list(mpars[midx[,i]==1, i])
> }
> dccfix = as.list(coef(fit1, "dcc"))
> spec2 = dccspec(uspec = multispec( vspec ), dccOrder = c(1,1),
> distribution = "mvnorm", fixed.pars = dccfix)
> # Fitted object
> sim1 = dccsim(fitORspec = fit1, n.sim = 1000, n.start = 100, m.sim = 2,
> startMethod = "unconditional",
>               rseed = NULL)
> print(sim1@msim$rseed)
> > 497782424 497782425 995564850
> sim2 = dccsim(fitORspec = fit1, n.sim = 1000, n.start = 100, m.sim = 2,
> startMethod = "unconditional",
>               rseed=sim1@msim$rseed)
>
> all.equal(sim1@msim$simX[[1]],sim2@msim$simX[[1]])
> > TRUE
> all.equal(sim1@msim$simX[[2]],sim2@msim$simX[[2]])
> > TRUE
> # Specification Object
> sims1 = dccsim(fitORspec = spec2, n.sim = 1000, n.start = 100, m.sim = 2,
> startMethod = "unconditional",
>               rseed = NULL, mexsimdata = NULL, vexsimdata = NULL, preQ =
> last(rcor(fit1, type = "Q"))[,,1],
>               Qbar = fit1@mfit$Qbar)
> sims2 = dccsim(fitORspec = spec2, n.sim = 1000, n.start = 100, m.sim = 2,
> startMethod = "unconditional",
>               rseed=sims1@msim$rseed, preQ = last(rcor(fit1, type =
> "Q"))[,,1],
>               Qbar = fit1@mfit$Qbar)
>
> all.equal(sims1@msim$simX[[1]],sims2@msim$simX[[1]])
> > TRUE
> all.equal(sims1@msim$simX[[2]],sims2@msim$simX[[2]])
> > TRUE
> ###############################
>
> Regards,
>
> Alexios
>
>
> On 1/23/18 11:34 AM, Josh Segal wrote:
>
>> Hi Alexios (and others),
>>
>> First, thank you for creating rmgarch, it's great.
>>
>> I'm trying to generate simulations using dccsim from a spec, and find that
>> when I specify a single value for rseed, the simulation results are not
>> reproducible (that is, each run gives different results).  I can avoid
>> this
>> issue by either using a fit instead of a spec or by specifying a vector of
>> m.sim rseeds.
>>
>> I did some digging in the code, and found in rdcc-main.R:
>> For fit, line 1224: rseed = c(rseed, (1:m.sim)*(rseed+1))
>> For spec, line 1493: rseed = c(rseed, as.integer(runif(m.sim, 1,
>> Sys.time())))
>>
>> I think this explains my results.  Seems like the latter is incorrect, and
>> should match the former, no?
>>
>> Thanks very much,
>> Josh
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions
>> should go.
>>
>>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Apparent bug in rmgarch

Josh Segal
A separate but related issue:

Fitting a VAR model with the varxfit function depends on a random seed via
runif() in the mlts() function - without the ability to set the seed.  This
creates reproducibility issues.  Perhaps an rseed argument should be made
available like some of the other functions in the package.

Thanks very much!

On Fri, Jan 26, 2018 at 10:45 AM Josh Segal <[hidden email]> wrote:

> I'll try that, thanks!
>
> On Wed, Jan 24, 2018 at 10:15 PM, alexios galanos <[hidden email]>
> wrote:
>
>> Hi Josh,
>>
>> Thanks for reporting this...I'll investigate further when time permits.
>>
>> In the meantime, if you let the rseed=NULL, the returned object will
>> include the seeds used which can then be passed back into the simulation
>> function to obtain reproducible results.
>>
>> Here is an example:
>> ###############################
>> library(rmgarch)
>> data(dji30retw)
>> Dat = dji30retw[, 1:3, drop = FALSE]
>> cnames = colnames(Dat)
>> uspec = ugarchspec(mean.model = list(armaOrder = c(2,1)), variance.model
>> = list(garchOrder = c(1,1), model = "eGARCH"), distribution.model = "norm")
>> spec1 = dccspec(uspec = multispec( replicate(3, uspec) ), dccOrder =
>> c(1,1), distribution = "mvnorm")
>> fit1 = dccfit(spec1, data = Dat, fit.control = list(eval.se=FALSE))
>> uspec = ugarchspec(mean.model = list(armaOrder = c(2,1)), variance.model
>> = list(garchOrder = c(1,1), model = "eGARCH"), distribution.model = "norm")
>> vspec = vector(mode = "list", length = 3)
>> midx = fit1@model$midx
>> mpars = fit1@model$mpars
>> # nasty, I know...will try to write some wrapper to perform this
>> automatically in future.
>> for(i in 1:3){
>>   vspec[[i]] = uspec
>>   setfixed(vspec[[i]])<-as.list(mpars[midx[,i]==1, i])
>> }
>> dccfix = as.list(coef(fit1, "dcc"))
>> spec2 = dccspec(uspec = multispec( vspec ), dccOrder = c(1,1),
>> distribution = "mvnorm", fixed.pars = dccfix)
>> # Fitted object
>> sim1 = dccsim(fitORspec = fit1, n.sim = 1000, n.start = 100, m.sim = 2,
>> startMethod = "unconditional",
>>               rseed = NULL)
>> print(sim1@msim$rseed)
>> > 497782424 497782425 995564850
>> sim2 = dccsim(fitORspec = fit1, n.sim = 1000, n.start = 100, m.sim = 2,
>> startMethod = "unconditional",
>>               rseed=sim1@msim$rseed)
>>
>> all.equal(sim1@msim$simX[[1]],sim2@msim$simX[[1]])
>> > TRUE
>> all.equal(sim1@msim$simX[[2]],sim2@msim$simX[[2]])
>> > TRUE
>> # Specification Object
>> sims1 = dccsim(fitORspec = spec2, n.sim = 1000, n.start = 100, m.sim = 2,
>> startMethod = "unconditional",
>>               rseed = NULL, mexsimdata = NULL, vexsimdata = NULL, preQ =
>> last(rcor(fit1, type = "Q"))[,,1],
>>               Qbar = fit1@mfit$Qbar)
>> sims2 = dccsim(fitORspec = spec2, n.sim = 1000, n.start = 100, m.sim = 2,
>> startMethod = "unconditional",
>>               rseed=sims1@msim$rseed, preQ = last(rcor(fit1, type =
>> "Q"))[,,1],
>>               Qbar = fit1@mfit$Qbar)
>>
>> all.equal(sims1@msim$simX[[1]],sims2@msim$simX[[1]])
>> > TRUE
>> all.equal(sims1@msim$simX[[2]],sims2@msim$simX[[2]])
>> > TRUE
>> ###############################
>>
>> Regards,
>>
>> Alexios
>>
>>
>> On 1/23/18 11:34 AM, Josh Segal wrote:
>>
>>> Hi Alexios (and others),
>>>
>>> First, thank you for creating rmgarch, it's great.
>>>
>>> I'm trying to generate simulations using dccsim from a spec, and find
>>> that
>>> when I specify a single value for rseed, the simulation results are not
>>> reproducible (that is, each run gives different results).  I can avoid
>>> this
>>> issue by either using a fit instead of a spec or by specifying a vector
>>> of
>>> m.sim rseeds.
>>>
>>> I did some digging in the code, and found in rdcc-main.R:
>>> For fit, line 1224: rseed = c(rseed, (1:m.sim)*(rseed+1))
>>> For spec, line 1493: rseed = c(rseed, as.integer(runif(m.sim, 1,
>>> Sys.time())))
>>>
>>> I think this explains my results.  Seems like the latter is incorrect,
>>> and
>>> should match the former, no?
>>>
>>> Thanks very much,
>>> Josh
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R questions
>>> should go.
>>>
>>>
>>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.