exponentially weighted linear regression

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

exponentially weighted linear regression

LosemindL
Hi all, Happy New Year!

Is there a function for exponentially weighted linear regression in R?

Usually, a linear regression is on a trunk of data...

And if I run linear regression on time series, I divide the time series
into "overlapped/rolling" windows and run linear regression on each rolling
chunk of data...

Is there a way to turn the rolling linear regression on the whole time
series into an exponentially weighted one, i.e. using a decay factor lambda
to give more weights to the newer observations...?

Are there packages in R which can do that?

Thanks a lot!

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

Zachary Mayer
Hi Michael,

R has lots of functions for exponential smoothing.
ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.ets>
in
the forecast package<http://cran.r-project.org/web/packages/forecast/index.html>
is
particularly useful.  Additionally, many models in R (including lm and glm)
have a weights argument.  You can come up with any weighting scheme you
wish, and pass it to your model as a weights argument.

There may be more useful references on the CRAN time-series task
view<http://cran.r-project.org/web/views/TimeSeries.html>
.

Regards,

Zach

On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:

> Hi all, Happy New Year!
>
> Is there a function for exponentially weighted linear regression in R?
>
> Usually, a linear regression is on a trunk of data...
>
> And if I run linear regression on time series, I divide the time series
> into "overlapped/rolling" windows and run linear regression on each rolling
> chunk of data...
>
> Is there a way to turn the rolling linear regression on the whole time
> series into an exponentially weighted one, i.e. using a decay factor lambda
> to give more weights to the newer observations...?
>
> Are there packages in R which can do that?
>
> Thanks a lot!
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

LosemindL
Thanks Zach.

The problem with these functions (e.g. lm or glm with weights argument) is
that they still do it on a divided trunk of data... i.e. block by block,
not the whole time series...

If we think about the exponential moving average estimate of volatility
with a decay factor lamda, it is actually on the whole time series... not
divided trunk of data.
Any thoughts?

On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]> wrote:

> Hi Michael,
>
> R has lots of functions for exponential smoothing.  ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.ets> in
> the forecast package<http://cran.r-project.org/web/packages/forecast/index.html> is
> particularly useful.  Additionally, many models in R (including lm and glm)
> have a weights argument.  You can come up with any weighting scheme you
> wish, and pass it to your model as a weights argument.
>
> There may be more useful references on the CRAN time-series task view<http://cran.r-project.org/web/views/TimeSeries.html>
> .
>
> Regards,
>
> Zach
>
>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:
>
>>  Hi all, Happy New Year!
>>
>> Is there a function for exponentially weighted linear regression in R?
>>
>> Usually, a linear regression is on a trunk of data...
>>
>> And if I run linear regression on time series, I divide the time series
>> into "overlapped/rolling" windows and run linear regression on each
>> rolling
>> chunk of data...
>>
>> Is there a way to turn the rolling linear regression on the whole time
>> series into an exponentially weighted one, i.e. using a decay factor
>> lambda
>> to give more weights to the newer observations...?
>>
>> Are there packages in R which can do that?
>>
>> Thanks a lot!
>>
>>        [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions
>> should go.
>>
>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

Zachary Mayer
I'm not following you.  What do you mean by "a divided trunk of data?"  As
far as I know, lm and glm will take as much data as you give them, until
you run out of memory.  Why not run the lm on the whole time series?

Perhaps you could post a reproducible example, so we can see what you're
trying to do.

On Thu, Jan 5, 2012 at 10:16 AM, Michael <[hidden email]> wrote:

>
> Thanks Zach.
>
> The problem with these functions (e.g. lm or glm with weights argument) is
> that they still do it on a divided trunk of data... i.e. block by block,
> not the whole time series...
>
> If we think about the exponential moving average estimate of volatility
> with a decay factor lamda, it is actually on the whole time series... not
> divided trunk of data.
> Any thoughts?
>
> On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]>wrote:
>
>> Hi Michael,
>>
>> R has lots of functions for exponential smoothing.  ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.ets> in
>> the forecast package<http://cran.r-project.org/web/packages/forecast/index.html> is
>> particularly useful.  Additionally, many models in R (including lm and glm)
>> have a weights argument.  You can come up with any weighting scheme you
>> wish, and pass it to your model as a weights argument.
>>
>> There may be more useful references on the CRAN time-series task view<http://cran.r-project.org/web/views/TimeSeries.html>
>> .
>>
>> Regards,
>>
>> Zach
>>
>>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:
>>
>>>  Hi all, Happy New Year!
>>>
>>> Is there a function for exponentially weighted linear regression in R?
>>>
>>> Usually, a linear regression is on a trunk of data...
>>>
>>> And if I run linear regression on time series, I divide the time series
>>> into "overlapped/rolling" windows and run linear regression on each
>>> rolling
>>> chunk of data...
>>>
>>> Is there a way to turn the rolling linear regression on the whole time
>>> series into an exponentially weighted one, i.e. using a decay factor
>>> lambda
>>> to give more weights to the newer observations...?
>>>
>>> Are there packages in R which can do that?
>>>
>>> Thanks a lot!
>>>
>>>        [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R questions
>>> should go.
>>>
>>
>>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

LosemindL
What I am looking for is some kind of "exponential moving average linear
regression"...

In a typical linear regression, we divide the time series into trunks...

For example, we run Linear Regression on the following trunks of time
series data (window size K):

1... K,
2... (K+1),
3... (K+2),
...
...
(N-K+1) ... N
----------------

Are you saying that I can use the exponentially weighted linear regression
on the time series as data become available?

1 ... K,
1 ... (K+1),
1 ... (K+2),
1 ... (K+3),
...
...
1 ... N

Then each time what should be my "exponentially decaying" weights?

----------------

Thanks a  lot!
On Thu, Jan 5, 2012 at 9:22 AM, Zachary Mayer <[hidden email]> wrote:

> I'm not following you.  What do you mean by "a divided trunk of data?"  As
> far as I know, lm and glm will take as much data as you give them, until
> you run out of memory.  Why not run the lm on the whole time series?
>
> Perhaps you could post a reproducible example, so we can see what you're
> trying to do.
>
>
> On Thu, Jan 5, 2012 at 10:16 AM, Michael <[hidden email]> wrote:
>
>>
>> Thanks Zach.
>>
>> The problem with these functions (e.g. lm or glm with weights argument)
>> is that they still do it on a divided trunk of data... i.e. block by block,
>> not the whole time series...
>>
>> If we think about the exponential moving average estimate of volatility
>> with a decay factor lamda, it is actually on the whole time series... not
>> divided trunk of data.
>> Any thoughts?
>>
>> On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]>wrote:
>>
>>> Hi Michael,
>>>
>>> R has lots of functions for exponential smoothing.  ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.ets> in
>>> the forecast package<http://cran.r-project.org/web/packages/forecast/index.html> is
>>> particularly useful.  Additionally, many models in R (including lm and glm)
>>> have a weights argument.  You can come up with any weighting scheme you
>>> wish, and pass it to your model as a weights argument.
>>>
>>> There may be more useful references on the CRAN time-series task view<http://cran.r-project.org/web/views/TimeSeries.html>
>>> .
>>>
>>> Regards,
>>>
>>> Zach
>>>
>>>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:
>>>
>>>>  Hi all, Happy New Year!
>>>>
>>>> Is there a function for exponentially weighted linear regression in R?
>>>>
>>>> Usually, a linear regression is on a trunk of data...
>>>>
>>>> And if I run linear regression on time series, I divide the time series
>>>> into "overlapped/rolling" windows and run linear regression on each
>>>> rolling
>>>> chunk of data...
>>>>
>>>> Is there a way to turn the rolling linear regression on the whole time
>>>> series into an exponentially weighted one, i.e. using a decay factor
>>>> lambda
>>>> to give more weights to the newer observations...?
>>>>
>>>> Are there packages in R which can do that?
>>>>
>>>> Thanks a lot!
>>>>
>>>>        [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>> -- Also note that this is not the r-help list where general R questions
>>>> should go.
>>>>
>>>
>>>
>>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

LosemindL
Here is one example from this link:

http://stats.stackexchange.com/questions/9931/exponentially-weighted-moving-linear-regression

I am not sure this is the best solution...

If you think of the exponential moving average of volatility(Risk Metrics
standard), that is more elegant)???

R> x <- 1:10    ## mean of this is 5.5
R> lm(x ~ 1)    ## regression on constant computes mean

Call:
lm(formula = x ~ 1)

Coefficients:
(Intercept)
        5.5

R> lm(x ~ 1, weights=0.9^(seq(10,1,by=-1)))

Call:
lm(formula = x ~ 1, weights = 0.9^(seq(10, 1, by = -1)))

Coefficients:
(Intercept)
       6.35

R>



On Thu, Jan 5, 2012 at 9:37 AM, Michael <[hidden email]> wrote:

> What I am looking for is some kind of "exponential moving average linear
> regression"...
>
> In a typical linear regression, we divide the time series into trunks...
>
> For example, we run Linear Regression on the following trunks of time
> series data (window size K):
>
> 1... K,
> 2... (K+1),
> 3... (K+2),
> ...
> ...
> (N-K+1) ... N
> ----------------
>
> Are you saying that I can use the exponentially weighted linear regression
> on the time series as data become available?
>
> 1 ... K,
> 1 ... (K+1),
> 1 ... (K+2),
> 1 ... (K+3),
> ...
> ...
> 1 ... N
>
> Then each time what should be my "exponentially decaying" weights?
>
> ----------------
>
> Thanks a  lot!
>  On Thu, Jan 5, 2012 at 9:22 AM, Zachary Mayer <[hidden email]>wrote:
>
>> I'm not following you.  What do you mean by "a divided trunk of data?"
>>  As far as I know, lm and glm will take as much data as you give them,
>> until you run out of memory.  Why not run the lm on the whole time series?
>>
>> Perhaps you could post a reproducible example, so we can see what you're
>> trying to do.
>>
>>
>> On Thu, Jan 5, 2012 at 10:16 AM, Michael <[hidden email]> wrote:
>>
>>>
>>> Thanks Zach.
>>>
>>> The problem with these functions (e.g. lm or glm with weights argument)
>>> is that they still do it on a divided trunk of data... i.e. block by block,
>>> not the whole time series...
>>>
>>> If we think about the exponential moving average estimate of volatility
>>> with a decay factor lamda, it is actually on the whole time series... not
>>> divided trunk of data.
>>> Any thoughts?
>>>
>>> On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]>wrote:
>>>
>>>> Hi Michael,
>>>>
>>>> R has lots of functions for exponential smoothing.  ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.ets> in
>>>> the forecast package<http://cran.r-project.org/web/packages/forecast/index.html> is
>>>> particularly useful.  Additionally, many models in R (including lm and glm)
>>>> have a weights argument.  You can come up with any weighting scheme you
>>>> wish, and pass it to your model as a weights argument.
>>>>
>>>> There may be more useful references on the CRAN time-series task view<http://cran.r-project.org/web/views/TimeSeries.html>
>>>> .
>>>>
>>>> Regards,
>>>>
>>>> Zach
>>>>
>>>>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:
>>>>
>>>>>  Hi all, Happy New Year!
>>>>>
>>>>> Is there a function for exponentially weighted linear regression in R?
>>>>>
>>>>> Usually, a linear regression is on a trunk of data...
>>>>>
>>>>> And if I run linear regression on time series, I divide the time series
>>>>> into "overlapped/rolling" windows and run linear regression on each
>>>>> rolling
>>>>> chunk of data...
>>>>>
>>>>> Is there a way to turn the rolling linear regression on the whole time
>>>>> series into an exponentially weighted one, i.e. using a decay factor
>>>>> lambda
>>>>> to give more weights to the newer observations...?
>>>>>
>>>>> Are there packages in R which can do that?
>>>>>
>>>>> Thanks a lot!
>>>>>
>>>>>        [[alternative HTML version deleted]]
>>>>>
>>>>> _______________________________________________
>>>>> [hidden email] mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>>> -- Also note that this is not the r-help list where general R
>>>>> questions should go.
>>>>>
>>>>
>>>>
>>>
>>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

riccardo visca
In reply to this post by Zachary Mayer
I think you should use a time varying parameter model (look at dlm package) or if you do not have time to learn try to smooth the predictors first then run the rolling window regression maybe robustified.

If you use the Kalman filter use an AR(1) for the parameter in the state equation like

yt = bt * Xt-1
bt = c * bt-1

c will do the smoothing (it is your (1-lambda))

From my point of view there is no cheap and dirty solution to this problem.




________________________________
 Da: Zachary Mayer <[hidden email]>
A: Michael <[hidden email]>
Cc: r-sig-finance <[hidden email]>
Inviato: Giovedì 5 Gennaio 2012 15:22
Oggetto: Re: [R-SIG-Finance] exponentially weighted linear regression

I'm not following you.  What do you mean by "a divided trunk of data?"  As
far as I know, lm and glm will take as much data as you give them, until
you run out of memory.  Why not run the lm on the whole time series?

Perhaps you could post a reproducible example, so we can see what you're
trying to do.

On Thu, Jan 5, 2012 at 10:16 AM, Michael <[hidden email]> wrote:

>
> Thanks Zach.
>
> The problem with these functions (e.g. lm or glm with weights argument) is
> that they still do it on a divided trunk of data... i.e. block by block,
> not the whole time series...
>
> If we think about the exponential moving average estimate of volatility
> with a decay factor lamda, it is actually on the whole time series... not
> divided trunk of data.
> Any thoughts?
>
> On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]>wrote:
>
>> Hi Michael,
>>
>> R has lots of functions for exponential smoothing.  ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.ets> in
>> the forecast package<http://cran.r-project.org/web/packages/forecast/index.html> is
>> particularly useful.  Additionally, many models in R (including lm and glm)
>> have a weights argument.  You can come up with any weighting scheme you
>> wish, and pass it to your model as a weights argument.
>>
>> There may be more useful references on the CRAN time-series task view<http://cran.r-project.org/web/views/TimeSeries.html>
>> .
>>
>> Regards,
>>
>> Zach
>>
>>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:
>>
[[elided Yahoo spam]]

>>>
>>> Is there a function for exponentially weighted linear regression in R?
>>>
>>> Usually, a linear regression is on a trunk of data...
>>>
>>> And if I run linear regression on time series, I divide the time series
>>> into "overlapped/rolling" windows and run linear regression on each
>>> rolling
>>> chunk of data...
>>>
>>> Is there a way to turn the rolling linear regression on the whole time
>>> series into an exponentially weighted one, i.e. using a decay factor
>>> lambda
>>> to give more weights to the newer observations...?
>>>
>>> Are there packages in R which can do that?
>>>
>>> Thanks a lot!
>>>
>>>        [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R questions
>>> should go.
>>>
>>
>>
>
    [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
        [[alternative HTML version deleted]]


_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

Zachary Mayer
In reply to this post by LosemindL
You run the lm on all available data at each step.  You have to
re-calculate the weights for the whole time series at each step.  It's up
to you to come up with a weighting scheme that makes sense to you.

Choose a decay factor (0,1], call it lambda. Your weighting scheme
is lambda^(n-t), where n is the length of your series and t is the
observation number.  If you had a time series with a 100 points, the first
weight would be lambda^100, and the last weight would be lambda^1.  Lower
values of lambda place more weight on recent observations.

On Thu, Jan 5, 2012 at 10:40 AM, Michael <[hidden email]> wrote:

>
>
> Here is one example from this link:
>
>
> http://stats.stackexchange.com/questions/9931/exponentially-weighted-moving-linear-regression
>
> I am not sure this is the best solution...
>
> If you think of the exponential moving average of volatility(Risk Metrics
> standard), that is more elegant)???
>
> R> x <- 1:10    ## mean of this is 5.5
> R> lm(x ~ 1)    ## regression on constant computes mean
>
> Call:
> lm(formula = x ~ 1)
>
> Coefficients:
> (Intercept)
>         5.5
>
> R> lm(x ~ 1, weights=0.9^(seq(10,1,by=-1)))
>
> Call:
> lm(formula = x ~ 1, weights = 0.9^(seq(10, 1, by = -1)))
>
> Coefficients:
> (Intercept)
>        6.35
>
> R>
>
>
>
> On Thu, Jan 5, 2012 at 9:37 AM, Michael <[hidden email]> wrote:
>
>> What I am looking for is some kind of "exponential moving average linear
>> regression"...
>>
>> In a typical linear regression, we divide the time series into trunks...
>>
>> For example, we run Linear Regression on the following trunks of time
>> series data (window size K):
>>
>> 1... K,
>> 2... (K+1),
>> 3... (K+2),
>> ...
>> ...
>> (N-K+1) ... N
>> ----------------
>>
>> Are you saying that I can use the exponentially weighted linear
>> regression on the time series as data become available?
>>
>> 1 ... K,
>> 1 ... (K+1),
>> 1 ... (K+2),
>> 1 ... (K+3),
>> ...
>> ...
>> 1 ... N
>>
>> Then each time what should be my "exponentially decaying" weights?
>>
>> ----------------
>>
>> Thanks a  lot!
>>  On Thu, Jan 5, 2012 at 9:22 AM, Zachary Mayer <[hidden email]>wrote:
>>
>>> I'm not following you.  What do you mean by "a divided trunk of data?"
>>>  As far as I know, lm and glm will take as much data as you give them,
>>> until you run out of memory.  Why not run the lm on the whole time series?
>>>
>>> Perhaps you could post a reproducible example, so we can see what you're
>>> trying to do.
>>>
>>>
>>> On Thu, Jan 5, 2012 at 10:16 AM, Michael <[hidden email]> wrote:
>>>
>>>>
>>>> Thanks Zach.
>>>>
>>>> The problem with these functions (e.g. lm or glm with weights argument)
>>>> is that they still do it on a divided trunk of data... i.e. block by block,
>>>> not the whole time series...
>>>>
>>>> If we think about the exponential moving average estimate of volatility
>>>> with a decay factor lamda, it is actually on the whole time series... not
>>>> divided trunk of data.
>>>> Any thoughts?
>>>>
>>>> On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]>wrote:
>>>>
>>>>> Hi Michael,
>>>>>
>>>>> R has lots of functions for exponential smoothing.  ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.ets> in
>>>>> the forecast package<http://cran.r-project.org/web/packages/forecast/index.html> is
>>>>> particularly useful.  Additionally, many models in R (including lm and glm)
>>>>> have a weights argument.  You can come up with any weighting scheme you
>>>>> wish, and pass it to your model as a weights argument.
>>>>>
>>>>> There may be more useful references on the CRAN time-series task view<http://cran.r-project.org/web/views/TimeSeries.html>
>>>>> .
>>>>>
>>>>> Regards,
>>>>>
>>>>> Zach
>>>>>
>>>>>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]>wrote:
>>>>>
>>>>>>  Hi all, Happy New Year!
>>>>>>
>>>>>> Is there a function for exponentially weighted linear regression in R?
>>>>>>
>>>>>> Usually, a linear regression is on a trunk of data...
>>>>>>
>>>>>> And if I run linear regression on time series, I divide the time
>>>>>> series
>>>>>> into "overlapped/rolling" windows and run linear regression on each
>>>>>> rolling
>>>>>> chunk of data...
>>>>>>
>>>>>> Is there a way to turn the rolling linear regression on the whole time
>>>>>> series into an exponentially weighted one, i.e. using a decay factor
>>>>>> lambda
>>>>>> to give more weights to the newer observations...?
>>>>>>
>>>>>> Are there packages in R which can do that?
>>>>>>
>>>>>> Thanks a lot!
>>>>>>
>>>>>>        [[alternative HTML version deleted]]
>>>>>>
>>>>>> _______________________________________________
>>>>>> [hidden email] mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>>>> -- Also note that this is not the r-help list where general R
>>>>>> questions should go.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

LosemindL
In reply to this post by riccardo visca
Okay there is a Kalma filter package in R...

But I am not sure
 yt = bt * Xt-1
bt = c * bt-1

is the Kalman filter?
On Thu, Jan 5, 2012 at 9:44 AM, riccardo visca <[hidden email]>wrote:

>  I think you should use a time varying parameter model (look at dlm
> package) or if you do not have time to learn try to smooth the predictors
> first then run the rolling window regression maybe robustified.
>
> If you use the Kalman filter use an AR(1) for the parameter in the state
> equation like
>
> yt = bt * Xt-1
> bt = c * bt-1
>
> c will do the smoothing (it is your (1-lambda))
>
> From my point of view there is no cheap and dirty solution to this problem.
>
>
>
>  ------------------------------
> *Da:* Zachary Mayer <[hidden email]>
> *A:* Michael <[hidden email]>
> *Cc:* r-sig-finance <[hidden email]>
> *Inviato:* Giovedì 5 Gennaio 2012 15:22
> *Oggetto:* Re: [R-SIG-Finance] exponentially weighted linear regression
>
> I'm not following you.  What do you mean by "a divided trunk of data?"  As
> far as I know, lm and glm will take as much data as you give them, until
> you run out of memory.  Why not run the lm on the whole time series?
>
> Perhaps you could post a reproducible example, so we can see what you're
> trying to do.
>
> On Thu, Jan 5, 2012 at 10:16 AM, Michael <[hidden email]> wrote:
>
> >
> > Thanks Zach.
> >
> > The problem with these functions (e.g. lm or glm with weights argument)
> is
> > that they still do it on a divided trunk of data... i.e. block by block,
> > not the whole time series...
> >
> > If we think about the exponential moving average estimate of volatility
> > with a decay factor lamda, it is actually on the whole time series... not
> > divided trunk of data.
> > Any thoughts?
> >
> > On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]
> >wrote:
> >
> >> Hi Michael,
> >>
> >> R has lots of functions for exponential smoothing.  ets<
> http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.ets> in
> >> the forecast package<
> http://cran.r-project.org/web/packages/forecast/index.html> is
>
> >> particularly useful.  Additionally, many models in R (including lm and
> glm)
> >> have a weights argument.  You can come up with any weighting scheme you
> >> wish, and pass it to your model as a weights argument.
> >>
> >> There may be more useful references on the CRAN time-series task view<
> http://cran.r-project.org/web/views/TimeSeries.html>
>
> >> .
> >>
> >> Regards,
> >>
> >> Zach
> >>
> >>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:
> >>
> >>>  Hi all, Happy New Year!
> >>>
> >>> Is there a function for exponentially weighted linear regression in R?
> >>>
> >>> Usually, a linear regression is on a trunk of data...
> >>>
> >>> And if I run linear regression on time series, I divide the time series
> >>> into "overlapped/rolling" windows and run linear regression on each
> >>> rolling
> >>> chunk of data...
> >>>
> >>> Is there a way to turn the rolling linear regression on the whole time
> >>> series into an exponentially weighted one, i.e. using a decay factor
> >>> lambda
> >>> to give more weights to the newer observations...?
> >>>
> >>> Are there packages in R which can do that?
> >>>
> >>> Thanks a lot!
> >>>
> >>>        [[alternative HTML version deleted]]
> >>>
> >>> _______________________________________________
> >>> [hidden email] mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> >>> -- Subscriber-posting only. If you want to post, subscribe first.
> >>> -- Also note that this is not the r-help list where general R questions
> >>> should go.
> >>>
> >>
> >>
> >
>
>     [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>
>
>
        [[alternative HTML version deleted]]


_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

Eric Zivot
In reply to this post by riccardo visca
It seems like this can be easily solved using rollApply or period.apply and
lm with exponentially declining weights (in the w option). BTW, regression
on exponentially weighted data is often called "discounted least squares"
and is what I would call a "poor man's" kalman filter. In fact there have
been a few papers in the engineering literature which shows that discounted
least squares is equivalent to a certain type of filtered estimated from a
state space regression model with time varying parameters. I've used
discounted least squares for modeling hedge fund data and found it to work
quite nicely. Also, the time varying parameter state-space model may work
well with a few explanatory variables (1 or 2) but it often does not perform
well if there are many (say 5+ predictors). For each regression parameter,
you have to estimate an AR(1) smoothing parameter plus a transition equation
error term. With many predictors, you run into numerical stability problems
very quickly and the likelihood function can have many local minima. For
this reason, discounted least squares is an attractive alternative. Finally,
it is very difficult to do model selection with the state space kalman
filter. How do you choose the variables to enter the regression equation? I
have not seen anyone do a systematic study of model selection in time
varying parameter models. Perhaps, this is where a Bayesian approach might
be useful. This is an important but neglected topic. I would be happy is
someone pointed me to some research on this topic.


Eric Zivot                                
Robert Richards Chaired Professor of Economics and Director of Outreach
Adjunct Professor of Finance                            
Adjunct Professor of Statistics
Adjunct Professor of Applied Mathematics
Department of Economics
Box 353330                  email:  [hidden email]
University of Washington    phone:  206-543-6715            
Seattle, WA 98195-3330
www:  http://faculty.washington.edu/ezivot                 




-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of riccardo visca
Sent: Thursday, January 05, 2012 7:44 AM
To: Zachary Mayer; Michael
Cc: r-sig-finance
Subject: Re: [R-SIG-Finance] exponentially weighted linear regression

I think you should use a time varying parameter model (look at dlm package)
or if you do not have time to learn try to smooth the predictors first then
run the rolling window regression maybe robustified.

If you use the Kalman filter use an AR(1) for the parameter in the state
equation like

yt = bt * Xt-1
bt = c * bt-1

c will do the smoothing (it is your (1-lambda))

>From my point of view there is no cheap and dirty solution to this problem.




________________________________
 Da: Zachary Mayer <[hidden email]>
A: Michael <[hidden email]>
Cc: r-sig-finance <[hidden email]>
Inviato: Giovedl 5 Gennaio 2012 15:22
Oggetto: Re: [R-SIG-Finance] exponentially weighted linear regression

I'm not following you.  What do you mean by "a divided trunk of data?"  As
far as I know, lm and glm will take as much data as you give them, until you
run out of memory.  Why not run the lm on the whole time series?

Perhaps you could post a reproducible example, so we can see what you're
trying to do.

On Thu, Jan 5, 2012 at 10:16 AM, Michael <[hidden email]> wrote:

>
> Thanks Zach.
>
> The problem with these functions (e.g. lm or glm with weights
> argument) is that they still do it on a divided trunk of data... i.e.
> block by block, not the whole time series...
>
> If we think about the exponential moving average estimate of
> volatility with a decay factor lamda, it is actually on the whole time
> series... not divided trunk of data.
> Any thoughts?
>
> On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]>wrote:
>
>> Hi Michael,
>>
>> R has lots of functions for exponential smoothing.  
>> ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.e
>> ts> in the forecast
>> package<http://cran.r-project.org/web/packages/forecast/index.html>
>> is particularly useful.  Additionally, many models in R (including lm and
glm) have a weights argument.  You can come up with any weighting scheme you
wish, and pass it to your model as a weights argument.

>>
>> There may be more useful references on the CRAN time-series task
>> view<http://cran.r-project.org/web/views/TimeSeries.html>
>> .
>>
>> Regards,
>>
>> Zach
>>
>>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:
>>
[[elided Yahoo spam]]

>>>
>>> Is there a function for exponentially weighted linear regression in R?
>>>
>>> Usually, a linear regression is on a trunk of data...
>>>
>>> And if I run linear regression on time series, I divide the time
>>> series into "overlapped/rolling" windows and run linear regression
>>> on each rolling chunk of data...
>>>
>>> Is there a way to turn the rolling linear regression on the whole
>>> time series into an exponentially weighted one, i.e. using a decay
>>> factor lambda to give more weights to the newer observations...?
>>>
>>> Are there packages in R which can do that?
>>>
>>> Thanks a lot!
>>>
>>>        [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R
>>> questions should go.
>>>
>>
>>
>

    [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions
should go.
        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

LosemindL
Interesting! In this "discounted LS“ model, how does one choose the
exponentially decaying weights then?

And also it still has the window-size for rolling windows...

So now you have two parameters ...

Originally I was thinking of using exponential EMA type of approach to
remove the window size... and only use one parameter the lamda - the decay
parameter...

Thanks!

On Mon, Jan 9, 2012 at 5:00 PM, Eric Zivot <[hidden email]> wrote:

> It seems like this can be easily solved using rollApply or period.apply and
> lm with exponentially declining weights (in the w option). BTW, regression
> on exponentially weighted data is often called "discounted least squares"
> and is what I would call a "poor man's" kalman filter. In fact there have
> been a few papers in the engineering literature which shows that discounted
> least squares is equivalent to a certain type of filtered estimated from a
> state space regression model with time varying parameters. I've used
> discounted least squares for modeling hedge fund data and found it to work
> quite nicely. Also, the time varying parameter state-space model may work
> well with a few explanatory variables (1 or 2) but it often does not
> perform
> well if there are many (say 5+ predictors). For each regression parameter,
> you have to estimate an AR(1) smoothing parameter plus a transition
> equation
> error term. With many predictors, you run into numerical stability problems
> very quickly and the likelihood function can have many local minima. For
> this reason, discounted least squares is an attractive alternative.
> Finally,
> it is very difficult to do model selection with the state space kalman
> filter. How do you choose the variables to enter the regression equation? I
> have not seen anyone do a systematic study of model selection in time
> varying parameter models. Perhaps, this is where a Bayesian approach might
> be useful. This is an important but neglected topic. I would be happy is
> someone pointed me to some research on this topic.
>
>
> Eric Zivot
> Robert Richards Chaired Professor of Economics and Director of Outreach
> Adjunct Professor of Finance
> Adjunct Professor of Statistics
> Adjunct Professor of Applied Mathematics
> Department of Economics
> Box 353330                  email:  [hidden email]
> University of Washington    phone:  206-543-6715
> Seattle, WA 98195-3330
> www:  http://faculty.washington.edu/ezivot
>
>
>
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of riccardo visca
> Sent: Thursday, January 05, 2012 7:44 AM
> To: Zachary Mayer; Michael
> Cc: r-sig-finance
> Subject: Re: [R-SIG-Finance] exponentially weighted linear regression
>
> I think you should use a time varying parameter model (look at dlm package)
> or if you do not have time to learn try to smooth the predictors first then
> run the rolling window regression maybe robustified.
>
> If you use the Kalman filter use an AR(1) for the parameter in the state
> equation like
>
> yt = bt * Xt-1
> bt = c * bt-1
>
> c will do the smoothing (it is your (1-lambda))
>
> From my point of view there is no cheap and dirty solution to this problem.
>
>
>
>
> ________________________________
>  Da: Zachary Mayer <[hidden email]>
> A: Michael <[hidden email]>
> Cc: r-sig-finance <[hidden email]>
> Inviato: Giovedl 5 Gennaio 2012 15:22
>  Oggetto: Re: [R-SIG-Finance] exponentially weighted linear regression
>
> I'm not following you.  What do you mean by "a divided trunk of data?"  As
> far as I know, lm and glm will take as much data as you give them, until
> you
> run out of memory.  Why not run the lm on the whole time series?
>
> Perhaps you could post a reproducible example, so we can see what you're
> trying to do.
>
> On Thu, Jan 5, 2012 at 10:16 AM, Michael <[hidden email]> wrote:
>
> >
> > Thanks Zach.
> >
> > The problem with these functions (e.g. lm or glm with weights
> > argument) is that they still do it on a divided trunk of data... i.e.
> > block by block, not the whole time series...
> >
> > If we think about the exponential moving average estimate of
> > volatility with a decay factor lamda, it is actually on the whole time
> > series... not divided trunk of data.
> > Any thoughts?
> >
> > On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <[hidden email]
> >wrote:
> >
> >> Hi Michael,
> >>
> >> R has lots of functions for exponential smoothing.
> >> ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.e
> >> ts> in the forecast
> >> package<http://cran.r-project.org/web/packages/forecast/index.html>
> >> is particularly useful.  Additionally, many models in R (including lm
> and
> glm) have a weights argument.  You can come up with any weighting scheme
> you
> wish, and pass it to your model as a weights argument.
> >>
> >> There may be more useful references on the CRAN time-series task
> >> view<http://cran.r-project.org/web/views/TimeSeries.html>
> >> .
> >>
> >> Regards,
> >>
> >> Zach
> >>
> >>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <[hidden email]> wrote:
> >>
> [[elided Yahoo spam]]
> >>>
> >>> Is there a function for exponentially weighted linear regression in R?
> >>>
> >>> Usually, a linear regression is on a trunk of data...
> >>>
> >>> And if I run linear regression on time series, I divide the time
> >>> series into "overlapped/rolling" windows and run linear regression
> >>> on each rolling chunk of data...
> >>>
> >>> Is there a way to turn the rolling linear regression on the whole
> >>> time series into an exponentially weighted one, i.e. using a decay
> >>> factor lambda to give more weights to the newer observations...?
> >>>
> >>> Are there packages in R which can do that?
> >>>
> >>> Thanks a lot!
> >>>
> >>>        [[alternative HTML version deleted]]
> >>>
> >>> _______________________________________________
> >>> [hidden email] mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> >>> -- Subscriber-posting only. If you want to post, subscribe first.
> >>> -- Also note that this is not the r-help list where general R
> >>> questions should go.
> >>>
> >>
> >>
> >
>
>    [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>        [[alternative HTML version deleted]]
>
>
>
        [[alternative HTML version deleted]]


_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: exponentially weighted linear regression

tomreilly
This post has NOT been accepted by the mailing list yet.
If you form the Transfer Function Model y(t)=W(B)*X(t)+[THETA(B)/PHI(B)]*a(t) the operator [THETA(B)/PHI(B)] is the "smoothing component". For examnple if PHI(B)=1.0 and THETA(B)=1-.5B this would imply a set of weights of .5,.25,.125,... . in this way you could provide the answer to optimizing the "weighted moving linear regression" rather than assuming it's form.
Tom Reilly www.autobox.com