Kalman filter for a time series

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Kalman filter for a time series

Staff
I found an example at
http://www.bearcave.com/finance/random_r_hacks/kalman_smooth.html shown
below.  But it seems the structSSM function has been removed from KFAS
library so it won't run.  Does anyone know how to fix the code so that it
runs?



library(KFAS)
library(tseries)
library(timeSeries)
library(zoo)
library(quantmod)

getDailyPrices = function( tickerSym, startDate, endDate )
{
  prices = get.hist.quote( instrument = tickerSym, start = startDate,
end = endDate,
                       quote="AdjClose", provider="yahoo",
                       compression="d",  quiet=T)

  prices.ts = ts(prices)
  return( prices.ts )
}

kalmanFilter = function( x )
{
  t = x
  if (class(t) != "ts") {
    t = ts(t)
  }
  ssModel = structSSM( y = t, distribution="Gaussian")
  ssFit = fitSSM(inits=c(0.5*log(var(t)), 0.5*log(var(t))), model = ssModel )
  kfs = KFS( ssFit$model, smoothing="state", nsim=length(t))
  vals = kfs$a
  lastVal = vals[ length(vals)]
  return(lastVal)
}

Start = "2011-01-01"
End   = "2012-12-31"
SandP = "^GSPC"

windowWidth = 20
tsLength = 100

SAndP.ts = getDailyPrices( SandP, Start, End )
SAndP.ts = SAndP.ts[1:tsLength]
SAndP.smoothed = rollapply( data=SAndP.ts, width=windowWidth, FUN=kalmanFilter)

par(mfrow=c(1,1))
prices = coredata( SAndP.ts[windowWidth:length(SAndP.ts)])
plot(prices, col="blue", type="l")
lines(coredata(SAndP.smoothed), col="magenta")
par(mfrow=c(1,1))

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Kalman filter for a time series

Spencer Graves-4


On 2017-07-29 11:26 PM, Staff wrote:
> I found an example at
> http://www.bearcave.com/finance/random_r_hacks/kalman_smooth.html


       That example is signed by "Ian Kaplan".  There's a box at the
bottom of the page for you to email him.


> shown
> below.  But it seems the structSSM function has been removed from KFAS
> library


       or it never was part of KFAS.  I don't know.


> so it won't run.  Does anyone know how to fix the code so that it
> runs?


       Have you tried the vignette with KFAS?


       Hope this helps.
       Spencer Graves

>
> library(KFAS)
> library(tseries)
> library(timeSeries)
> library(zoo)
> library(quantmod)
>
> getDailyPrices = function( tickerSym, startDate, endDate )
> {
>    prices = get.hist.quote( instrument = tickerSym, start = startDate,
> end = endDate,
>                         quote="AdjClose", provider="yahoo",
>                         compression="d",  quiet=T)
>
>    prices.ts = ts(prices)
>    return( prices.ts )
> }
>
> kalmanFilter = function( x )
> {
>    t = x
>    if (class(t) != "ts") {
>      t = ts(t)
>    }
>    ssModel = structSSM( y = t, distribution="Gaussian")
>    ssFit = fitSSM(inits=c(0.5*log(var(t)), 0.5*log(var(t))), model = ssModel )
>    kfs = KFS( ssFit$model, smoothing="state", nsim=length(t))
>    vals = kfs$a
>    lastVal = vals[ length(vals)]
>    return(lastVal)
> }
>
> Start = "2011-01-01"
> End   = "2012-12-31"
> SandP = "^GSPC"
>
> windowWidth = 20
> tsLength = 100
>
> SAndP.ts = getDailyPrices( SandP, Start, End )
> SAndP.ts = SAndP.ts[1:tsLength]
> SAndP.smoothed = rollapply( data=SAndP.ts, width=windowWidth, FUN=kalmanFilter)
>
> par(mfrow=c(1,1))
> prices = coredata( SAndP.ts[windowWidth:length(SAndP.ts)])
> plot(prices, col="blue", type="l")
> lines(coredata(SAndP.smoothed), col="magenta")
> par(mfrow=c(1,1))
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Kalman filter for a time series

Roy Mendelssohn - NOAA Federal
In reply to this post by Staff
> structSSM

Is no longer part of KFAS.  All you needed to do was:

library(KFAS)
?KFAS

and you would have seen that if you went to the index.  A structural state space model is now built up from its components,  much like in LM.   Look at;

?SSModel

-Roy

> On Jul 29, 2017, at 9:26 PM, Staff <[hidden email]> wrote:
>
> I found an example at
> http://www.bearcave.com/finance/random_r_hacks/kalman_smooth.html shown
> below.  But it seems the structSSM function has been removed from KFAS
> library so it won't run.  Does anyone know how to fix the code so that it
> runs?
>
>
>
> library(KFAS)
> library(tseries)
> library(timeSeries)
> library(zoo)
> library(quantmod)
>
> getDailyPrices = function( tickerSym, startDate, endDate )
> {
>  prices = get.hist.quote( instrument = tickerSym, start = startDate,
> end = endDate,
>                       quote="AdjClose", provider="yahoo",
>                       compression="d",  quiet=T)
>
>  prices.ts = ts(prices)
>  return( prices.ts )
> }
>
> kalmanFilter = function( x )
> {
>  t = x
>  if (class(t) != "ts") {
>    t = ts(t)
>  }
>  ssModel = structSSM( y = t, distribution="Gaussian")
>  ssFit = fitSSM(inits=c(0.5*log(var(t)), 0.5*log(var(t))), model = ssModel )
>  kfs = KFS( ssFit$model, smoothing="state", nsim=length(t))
>  vals = kfs$a
>  lastVal = vals[ length(vals)]
>  return(lastVal)
> }
>
> Start = "2011-01-01"
> End   = "2012-12-31"
> SandP = "^GSPC"
>
> windowWidth = 20
> tsLength = 100
>
> SAndP.ts = getDailyPrices( SandP, Start, End )
> SAndP.ts = SAndP.ts[1:tsLength]
> SAndP.smoothed = rollapply( data=SAndP.ts, width=windowWidth, FUN=kalmanFilter)
>
> par(mfrow=c(1,1))
> prices = coredata( SAndP.ts[windowWidth:length(SAndP.ts)])
> plot(prices, col="blue", type="l")
> lines(coredata(SAndP.smoothed), col="magenta")
> par(mfrow=c(1,1))
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: [hidden email] www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Kalman filter for a time series

Staff
Exactly.  I'm doing:

x=c(1:length(t))
ssModel = SSModel( t~x, distribution="gaussian",H=NA)

but it is not plotting the Kalman filter line and not giving any errors.
I'm not sure what more the model needs?



On Sun, Jul 30, 2017 at 9:17 AM, Roy Mendelssohn - NOAA Federal <
[hidden email]> wrote:

> > structSSM
>
> Is no longer part of KFAS.  All you needed to do was:
>
> library(KFAS)
> ?KFAS
>
> and you would have seen that if you went to the index.  A structural state
> space model is now built up from its components,  much like in LM.   Look
> at;
>
> ?SSModel
>
> -Roy
>
> > On Jul 29, 2017, at 9:26 PM, Staff <[hidden email]> wrote:
> >
> > I found an example at
> > http://www.bearcave.com/finance/random_r_hacks/kalman_smooth.html shown
> > below.  But it seems the structSSM function has been removed from KFAS
> > library so it won't run.  Does anyone know how to fix the code so that it
> > runs?
> >
> >
> >
> > library(KFAS)
> > library(tseries)
> > library(timeSeries)
> > library(zoo)
> > library(quantmod)
> >
> > getDailyPrices = function( tickerSym, startDate, endDate )
> > {
> >  prices = get.hist.quote( instrument = tickerSym, start = startDate,
> > end = endDate,
> >                       quote="AdjClose", provider="yahoo",
> >                       compression="d",  quiet=T)
> >
> >  prices.ts = ts(prices)
> >  return( prices.ts )
> > }
> >
> > kalmanFilter = function( x )
> > {
> >  t = x
> >  if (class(t) != "ts") {
> >    t = ts(t)
> >  }
> >  ssModel = structSSM( y = t, distribution="Gaussian")
> >  ssFit = fitSSM(inits=c(0.5*log(var(t)), 0.5*log(var(t))), model =
> ssModel )
> >  kfs = KFS( ssFit$model, smoothing="state", nsim=length(t))
> >  vals = kfs$a
> >  lastVal = vals[ length(vals)]
> >  return(lastVal)
> > }
> >
> > Start = "2011-01-01"
> > End   = "2012-12-31"
> > SandP = "^GSPC"
> >
> > windowWidth = 20
> > tsLength = 100
> >
> > SAndP.ts = getDailyPrices( SandP, Start, End )
> > SAndP.ts = SAndP.ts[1:tsLength]
> > SAndP.smoothed = rollapply( data=SAndP.ts, width=windowWidth,
> FUN=kalmanFilter)
> >
> > par(mfrow=c(1,1))
> > prices = coredata( SAndP.ts[windowWidth:length(SAndP.ts)])
> > plot(prices, col="blue", type="l")
> > lines(coredata(SAndP.smoothed), col="magenta")
> > par(mfrow=c(1,1))
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: [hidden email] www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK
> Jr.
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Kalman filter for a time series

David Winsemius
In reply to this post by Spencer Graves-4

> On Jul 30, 2017, at 5:10 AM, Spencer Graves <[hidden email]> wrote:
>
>
>
> On 2017-07-29 11:26 PM, Staff wrote:
>> I found an example at
>> http://www.bearcave.com/finance/random_r_hacks/kalman_smooth.html
>
>      That example is signed by "Ian Kaplan".  There's a box at the bottom of the page for you to email him.
>
>> shown
>> below.  But it seems the structSSM function has been removed from KFAS
>> library
>
>      or it never was part of KFAS.  I don't know.

It was part of KFAS. Searching with Google finds both the code as well as postings to Rhelp and Freelancer.com with requests (some for pay) to write replacements.

>
>> so it won't run.  Does anyone know how to fix the code so that it
>> runs?
>
>      Have you tried the vignette with KFAS?

Indeed. The section on structural time series implies that the functionality of the structSSM function has been replaced with three functions: SSMtrend, SSMcycle, and SSMseasonal.

If you (meaning Staff, not Spencer) are not up to the task of making the transition to the new version of KFAS, then perhaps you should hire a consultant who can provide both code and sufficient documentation that you will be able to understand what is "under the hood".

Best;
David.
>
>
>      Hope this helps.
>      Spencer Graves
>>
>> library(KFAS)
>> library(tseries)
snipped
>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________________________

David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.'   -Gehm's Corollary to Clarke's Third Law

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Kalman filter for a time series

Staff
Thanks David, that's the clue I needed.  Since x monotonically increases,
all I needed to do was:

ssModel = SSModel( t ~ SSMtrend(degree=1, Q=matrix(NA)), H=matrix(NA),
distribution="gaussian")





On Sun, Jul 30, 2017 at 10:11 AM, David Winsemius <[hidden email]>
wrote:

>
> > On Jul 30, 2017, at 5:10 AM, Spencer Graves <spencer.graves@
> effectivedefense.org> wrote:
> >
> >
> >
> > On 2017-07-29 11:26 PM, Staff wrote:
> >> I found an example at
> >> http://www.bearcave.com/finance/random_r_hacks/kalman_smooth.html
> >
> >      That example is signed by "Ian Kaplan".  There's a box at the
> bottom of the page for you to email him.
> >
> >> shown
> >> below.  But it seems the structSSM function has been removed from KFAS
> >> library
> >
> >      or it never was part of KFAS.  I don't know.
>
> It was part of KFAS. Searching with Google finds both the code as well as
> postings to Rhelp and Freelancer.com with requests (some for pay) to write
> replacements.
>
> >
> >> so it won't run.  Does anyone know how to fix the code so that it
> >> runs?
> >
> >      Have you tried the vignette with KFAS?
>
> Indeed. The section on structural time series implies that the
> functionality of the structSSM function has been replaced with three
> functions: SSMtrend, SSMcycle, and SSMseasonal.
>
> If you (meaning Staff, not Spencer) are not up to the task of making the
> transition to the new version of KFAS, then perhaps you should hire a
> consultant who can provide both code and sufficient documentation that you
> will be able to understand what is "under the hood".
>
> Best;
> David.
> >
> >
> >      Hope this helps.
> >      Spencer Graves
> >>
> >> library(KFAS)
> >> library(tseries)
> snipped
> >>
> >>      [[alternative HTML version deleted]]
> >>
> >> ______________________________________________
>
> David Winsemius
> Alameda, CA, USA
>
> 'Any technology distinguishable from magic is insufficiently advanced.'
>  -Gehm's Corollary to Clarke's Third Law
>
>
>
>
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.