Evaluating/comparing dynamic linear model

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Evaluating/comparing dynamic linear model

Robert A'gata
Hi,

I have two DLM model specifications (x[t] and y[t] are univariate):

MODEL1:
y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2)
b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2)

MODEL2:
y[t] = a[t]+e[t], e[t] ~ N(0,v2^2)
a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2)

I run the filter through data recursively to obtain state variables
for each model. However, how do I know if b[t]x[t] in MODEL1 is
different from MODEL2? In other words, how do I know if x[t] makes a
difference in explaining dynamic of y[t]?

Another question is that how do I compare MODEL1 and MODEL2? From
model specification point of view, how can one say that MODEL1 is
better than MODEL2? Any suggestion/reference would be greatly
appreciated. Thank you.

ac

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: [R-SIG-Finance] Evaluating/comparing dynamic linear model

Erb Philipp (erbp)
What kind of filter are you using? Since your models are expressed in state space form I suggest that you fit your models by maximizing the log likelihood function of the Kalman filter output (see e.g. FKF-package). Using the obtained log likelihood values you might perform a likelihood ratio test to test the hypothesis whether model 1 explains yt "better" than model 2.

HTH, Phil


-----Ursprüngliche Nachricht-----
Von: [hidden email] [mailto:[hidden email]] Im Auftrag von R_help Help
Gesendet: Donnerstag, 8. Oktober 2009 02:55
An: [hidden email]; [hidden email]
Betreff: [R-SIG-Finance] Evaluating/comparing dynamic linear model

Hi,

I have two DLM model specifications (x[t] and y[t] are univariate):

MODEL1:
y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2)
b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2)

MODEL2:
y[t] = a[t]+e[t], e[t] ~ N(0,v2^2)
a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2)

I run the filter through data recursively to obtain state variables
for each model. However, how do I know if b[t]x[t] in MODEL1 is
different from MODEL2? In other words, how do I know if x[t] makes a
difference in explaining dynamic of y[t]?

Another question is that how do I compare MODEL1 and MODEL2? From
model specification point of view, how can one say that MODEL1 is
better than MODEL2? Any suggestion/reference would be greatly
appreciated. Thank you.

ac

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Evaluating/comparing dynamic linear model

Giovanni Petris

The standard asymptotic theory of likelihood ratio tests assumes that
you are testing a submodel, which is not the case here. Moreover, even
when testing submodels, there are other assumptions that often are not
met in the case of DLMs - the typical example being hypothesised
values on the boundary of the parameter space.

An informal, but commonly used approach to compare DLMs is to compare
their predictive performance. There are many ways to measure it. You
can find some of them in the book "dynamic linear models with R".

BTW, in the package dlm you find functions for MLE, forecasting,
one-step-ahead forecast errors (which is what you need to compute
predictive performance), and more...

HTH,
Giovanni


> Date: Thu, 08 Oct 2009 08:32:21 +0200
> From: "Erb Philipp (erbp)" <[hidden email]>
> Sender: [hidden email]
> Precedence: list
> Thread-topic: [R-SIG-Finance] Evaluating/comparing dynamic linear model
> Thread-index: AcpHsn7VAX05YB13ShmB34Y7FCJyDAALTuuA
>
> What kind of filter are you using? Since your models are expressed
> in state space form I suggest that you fit your models by maximizing
> the log likelihood function of the Kalman filter output (see
> e.g. FKF-package). Using the obtained log likelihood values you
> might perform a likelihood ratio test to test the hypothesis whether
> model 1 explains yt "better" than model 2.
>
> HTH, Phil
>
>
> -----Ursprüngliche Nachricht-----
> Von: [hidden email] [mailto:[hidden email]] Im Auftrag von R_help Help
> Gesendet: Donnerstag, 8. Oktober 2009 02:55
> An: [hidden email]; [hidden email]
> Betreff: [R-SIG-Finance] Evaluating/comparing dynamic linear model
>
> Hi,
>
> I have two DLM model specifications (x[t] and y[t] are univariate):
>
> MODEL1:
> y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2)
> b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2)
>
> MODEL2:
> y[t] = a[t]+e[t], e[t] ~ N(0,v2^2)
> a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2)
>
> I run the filter through data recursively to obtain state variables
> for each model. However, how do I know if b[t]x[t] in MODEL1 is
> different from MODEL2? In other words, how do I know if x[t] makes a
> difference in explaining dynamic of y[t]?
>
> Another question is that how do I compare MODEL1 and MODEL2? From
> model specification point of view, how can one say that MODEL1 is
> better than MODEL2? Any suggestion/reference would be greatly
> appreciated. Thank you.
>
> ac
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Evaluating/comparing dynamic linear model

Erb Philipp (erbp)
Apologies for my overhasty and thus inaccurate reply. I of course assumed that model2 is a submodel of model1. But thanks to Giovanni: a closer look at your equations shows that this is only the case if at == bt (and of course same innovations/ disturbances et, etat) and xt e.g. something like xt == theta0 * zt + 1. In that case you are allowed to perform a likelihood ratio test to test the hypothesis theta0 == 0. If model2 is included in model1 likelihood ratio test might be more convenient than comparing the predictive performance.


-----Original Message-----
From: Giovanni Petris [mailto:[hidden email]]
Sent: Thu 10/8/2009 3:55 PM
To: Erb Philipp (erbp)
Cc: [hidden email]; [hidden email]
Subject: Re: [R] Evaluating/comparing dynamic linear model
 

The standard asymptotic theory of likelihood ratio tests assumes that
you are testing a submodel, which is not the case here. Moreover, even
when testing submodels, there are other assumptions that often are not
met in the case of DLMs - the typical example being hypothesised
values on the boundary of the parameter space.

An informal, but commonly used approach to compare DLMs is to compare
their predictive performance. There are many ways to measure it. You
can find some of them in the book "dynamic linear models with R".

BTW, in the package dlm you find functions for MLE, forecasting,
one-step-ahead forecast errors (which is what you need to compute
predictive performance), and more...

HTH,
Giovanni


> Date: Thu, 08 Oct 2009 08:32:21 +0200
> From: "Erb Philipp (erbp)" <[hidden email]>
> Sender: [hidden email]
> Precedence: list
> Thread-topic: [R-SIG-Finance] Evaluating/comparing dynamic linear model
> Thread-index: AcpHsn7VAX05YB13ShmB34Y7FCJyDAALTuuA
>
> What kind of filter are you using? Since your models are expressed
> in state space form I suggest that you fit your models by maximizing
> the log likelihood function of the Kalman filter output (see
> e.g. FKF-package). Using the obtained log likelihood values you
> might perform a likelihood ratio test to test the hypothesis whether
> model 1 explains yt "better" than model 2.
>
> HTH, Phil
>
>
> -----Ursprüngliche Nachricht-----
> Von: [hidden email] [mailto:[hidden email]] Im Auftrag von R_help Help
> Gesendet: Donnerstag, 8. Oktober 2009 02:55
> An: [hidden email]; [hidden email]
> Betreff: [R-SIG-Finance] Evaluating/comparing dynamic linear model
>
> Hi,
>
> I have two DLM model specifications (x[t] and y[t] are univariate):
>
> MODEL1:
> y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2)
> b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2)
>
> MODEL2:
> y[t] = a[t]+e[t], e[t] ~ N(0,v2^2)
> a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2)
>
> I run the filter through data recursively to obtain state variables
> for each model. However, how do I know if b[t]x[t] in MODEL1 is
> different from MODEL2? In other words, how do I know if x[t] makes a
> difference in explaining dynamic of y[t]?
>
> Another question is that how do I compare MODEL1 and MODEL2? From
> model specification point of view, how can one say that MODEL1 is
> better than MODEL2? Any suggestion/reference would be greatly
> appreciated. Thank you.
>
> ac
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.