a question on autocorrelation acf

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

a question on autocorrelation acf

zhenjiang xu
Hi R users,

where can I find the equations used by acf function to calculate
autocorrelation? I think I misunderstand acf. Doesn't acf use following
equation to calculate autocorrelation?
[image: R(\tau) = \frac{\operatorname{E}[(X_t - \mu)(X_{t+\tau} -
\mu)]}{\sigma^2}\, ,]
If it does, then the autocorrelation of a sine function should give a
cosine; however, the following code gives a cosine-shape function with its
magnitude decreasing along the lag.
x = c(1:500)
x = x/10
x = sin(x)
acf(x, type='correlation', lag.max=length(x)-1)

--
Best,
Zhenjiang

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: a question on autocorrelation acf

Duncan Murdoch-2
On 29/04/2010 6:22 PM, zhenjiang xu wrote:
> Hi R users,
>
> where can I find the equations used by acf function to calculate
> autocorrelation?

See the reference listed in ?acf.

Duncan Murdoch

>  I think I misunderstand acf. Doesn't acf use following
> equation to calculate autocorrelation?
> [image: R(\tau) = \frac{\operatorname{E}[(X_t - \mu)(X_{t+\tau} -
> \mu)]}{\sigma^2}\, ,]
> If it does, then the autocorrelation of a sine function should give a
> cosine; however, the following code gives a cosine-shape function with its
> magnitude decreasing along the lag.
> x = c(1:500)
> x = x/10
> x = sin(x)
> acf(x, type='correlation', lag.max=length(x)-1)
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: a question on autocorrelation acf

zhenjiang xu
Thanks, Duncan, but there are no reference in ?acf. The only probably
related stuff is

"Author(s):

     Original: Paul Gilbert, Martyn Plummer. Extensive modifications
     and univariate case of 'pacf' by B.D. Ripley."

And I didn't find anything with google search of it.


On Thu, Apr 29, 2010 at 7:08 PM, Duncan Murdoch <[hidden email]>wrote:

> On 29/04/2010 6:22 PM, zhenjiang xu wrote:
>
>> Hi R users,
>>
>> where can I find the equations used by acf function to calculate
>> autocorrelation?
>>
>
> See the reference listed in ?acf.
>
> Duncan Murdoch
>
>
>   I think I misunderstand acf. Doesn't acf use following
>> equation to calculate autocorrelation?
>> [image: R(\tau) = \frac{\operatorname{E}[(X_t - \mu)(X_{t+\tau} -
>> \mu)]}{\sigma^2}\, ,]
>> If it does, then the autocorrelation of a sine function should give a
>> cosine; however, the following code gives a cosine-shape function with its
>> magnitude decreasing along the lag.
>> x = c(1:500)
>> x = x/10
>> x = sin(x)
>> acf(x, type='correlation', lag.max=length(x)-1)
>>
>>
>>
>
>


--
Best,
Zhenjiang

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: a question on autocorrelation acf

John Ramey
I think you are Googling the wrong "reference."  Note in ?acf the following:

References:

     Venables, W. N. and Ripley, B. D. (2002) _Modern Applied
     Statistics with S_.  Fourth Edition.  Springer-Verlag.

     (This contains the exact definitions used.)

On Fri, Apr 30, 2010 at 10:42 AM, zhenjiang xu <[hidden email]> wrote:

> Thanks, Duncan, but there are no reference in ?acf. The only probably
> related stuff is
>
> "Author(s):
>
>     Original: Paul Gilbert, Martyn Plummer. Extensive modifications
>     and univariate case of 'pacf' by B.D. Ripley."
>
> And I didn't find anything with google search of it.
>
>
> On Thu, Apr 29, 2010 at 7:08 PM, Duncan Murdoch <[hidden email]>wrote:
>
>> On 29/04/2010 6:22 PM, zhenjiang xu wrote:
>>
>>> Hi R users,
>>>
>>> where can I find the equations used by acf function to calculate
>>> autocorrelation?
>>>
>>
>> See the reference listed in ?acf.
>>
>> Duncan Murdoch
>>
>>
>>   I think I misunderstand acf. Doesn't acf use following
>>> equation to calculate autocorrelation?
>>> [image: R(\tau) = \frac{\operatorname{E}[(X_t - \mu)(X_{t+\tau} -
>>> \mu)]}{\sigma^2}\, ,]
>>> If it does, then the autocorrelation of a sine function should give a
>>> cosine; however, the following code gives a cosine-shape function with its
>>> magnitude decreasing along the lag.
>>> x = c(1:500)
>>> x = x/10
>>> x = sin(x)
>>> acf(x, type='correlation', lag.max=length(x)-1)
>>>
>>>
>>>
>>
>>
>
>
> --
> Best,
> Zhenjiang
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



--
John A. Ramey, M.S.
Ph.D. Candidate
Department of Statistics
Baylor University

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: a question on autocorrelation acf

Bazman76
Hi there,

I am wondering how R calculates the acf too?

I have a data set of approx 1500 returns when I calculate the lag 1 autocorrelation in excel I get a value of -0.4 but in R its approximately -0.18?

I have cross checked with PC give and PC give agrees with excel?

I'm sure its just some kind of scaling but it would be nice to resolve this discrepency!

Baz
Reply | Threaded
Open this post in threaded view
|

Re: a question on autocorrelation acf

Michael Weylandt
Did you check the MASS reference given above in the thread? If you
want to see the source, it's here:
http://svn.r-project.org/R/trunk/src/library/stats/src/filter.c (best
I can tell)

Michael

On Mon, Dec 5, 2011 at 2:24 PM, Bazman76 <[hidden email]> wrote:

> Hi there,
>
> I am wondering how R calculates the acf too?
>
> I have a data set of approx 1500 returns when I calculate the lag 1
> autocorrelation in excel I get a value of -0.4 but in R its approximately
> -0.18?
>
> I have cross checked with PC give and PC give agrees with excel?
>
> I'm sure its just some kind of scaling but it would be nice to resolve this
> discrepency!
>
> Baz
>
> --
> View this message in context: http://r.789695.n4.nabble.com/a-question-on-autocorrelation-acf-tp2076280p4161818.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: a question on autocorrelation acf

Bazman76
Hi there,

I have now cheked the results against pc give and the excel add-in poptools.

Poptools and pc give get the same answer but R is quite different especially for the acf and pacf()?

I looked at the book you recommended on p390 itshows the formulas and they look pretty standard.

However looking at the code that you sent the acf function does not seem to be being calculated as shown in the book?

At no point is the series mean calculated? unless the variable x is already demeaned in some way ie. there is some preprocessing that I need to see to fully understand?

Reply | Threaded
Open this post in threaded view
|

Re: a question on autocorrelation acf

Michael Weylandt
The code provided was the hard-to-find C part: there's also some R
code involved that you can get in your console by typing acf without
parentheses.

I'm inclined to believe V&R over any Excel implementation (and I don't
know what "pc" is) but perhaps you can provide a (small) data-set
using the dput() function and say what you are expecting to get from
acf() instead.

Michael

On Mon, Dec 5, 2011 at 8:32 PM, Bazman76 <[hidden email]> wrote:

> Hi there,
>
> I have now cheked the results against pc give and the excel add-in poptools.
>
> Poptools and pc give get the same answer but R is quite different especially
> for the acf and pacf()?
>
> I looked at the book you recommended on p390 itshows the formulas and they
> look pretty standard.
>
> However looking at the code that you sent the acf function does not seem to
> be being calculated as shown in the book?
>
> At no point is the series mean calculated? unless the variable x is already
> demeaned in some way ie. there is some preprocessing that I need to see to
> fully understand?
>
>
>
> --
> View this message in context: http://r.789695.n4.nabble.com/a-question-on-autocorrelation-acf-tp2076280p4163021.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: a question on autocorrelation acf

Bazman76
R_example.xlsx

Hi there,

I attach an excel file which I use to produce the data.

It simulates a simple AR(1) process y_t=0.5y_{t-1}+z_t.

In column E I have cut and paste values so that we can compare like with like.

When I run the acf() on these values, it shows 3 significant lags.

When I run the pacf() it shows one very strong correlation of 0.3 at lag 1, and a smaller one at lag 2 of around 0.05.

Now according to the theory the pacf() results should give the correct exponents for the lags.
This is a pure AR(1) process so the results should be 0.5 on lag 1 and statistically insignificant else where?

I have tried if for several realisations of the white noise term and the results are qualatively similar and is disagreement with the theory.


Kind Regards

Baz