Identify period length of time series automatically?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Identify period length of time series automatically?

Rainer M Krug-6
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi

I have 10.000 simulations for a sensitivity analysis. I have done a few
sensitivity analysis for different response variables already,
but now, as most of the simulations (if not all) show some cyclic
behaviour, see how the independent input parameter influence the
frequency of the cyclic changes and "how cyclic" they actually are.

So effectively, I have 39 values, and I want to identify automatically
the frequency / period length of the series and a kind of a measure on
"how cyclic" the series is.

How can I do that automatically without individual checking? I do not
want to do an eyeball assessment for 10.000 time series....

Thanks,

Rainer

- --
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      [hidden email]

Skype:      RMkrug
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2mvnMACgkQoYgNqgF2egoseACdG0VDzA2XpOTxshcIMOplVKsY
i5gAmwSMTuNdAwO74200RlzL9Wu0aJ05
=kh0m
-----END PGP SIGNATURE-----

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Identify period length of time series automatically?

Mike Marchywka










----------------------------------------

> Date: Thu, 14 Apr 2011 11:29:23 +0200
> From: [hidden email]
> To: [hidden email]
> Subject: [R] Identify period length of time series automatically?
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi
>
> I have 10.000 simulations for a sensitivity analysis. I have done a few
> sensitivity analysis for different response variables already,
> but now, as most of the simulations (if not all) show some cyclic
> behaviour, see how the independent input parameter influence the
> frequency of the cyclic changes and "how cyclic" they actually are.
>
> So effectively, I have 39 values, and I want to identify automatically
> the frequency / period length of the series and a kind of a measure on
> "how cyclic" the series is.

Probably google "Digital Signal Processing" or Fourier transform.
>From this, you resolve your time series into sinusoids of various components
and you can separate peaks in line spectra from background noise.
Depending on what you consider to be "cyclic" the analysis details
will vary. If you look at things like amplitude and frequncy modulation
of one sine wave with another and various relationships between carrier and
modulation frequency, you can get some ideas of what to look for in spectra.

Alternatively, you can try to define exactly what you mean by "cyclic"
and maybe make a better transform that discriminates that from acyclic
but offhand I would suggest FFT and various tests on the spectra.


Just off hand I'm not sure that 39 points would be a lot to go on
but you can simulate some examples in R quite easily if you know
what the data looks like in various cases you think may exist.





>
> How can I do that automatically without individual checking? I do not
> want to do an eyeball assessment for 10.000 time series....
>
> Thanks,
>
> Rainer
>
> - --
> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
> Biology, UCT), Dipl. Phys. (Germany)
>
> Centre of Excellence for Invasion Biology
> Stellenbosch University
> South Africa

     
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Identify period length of time series automatically?

Rainer M Krug-6
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 14/04/11 11:57, Mike Marchywka wrote:

>
>
>
>
>
>
>
>
>
>
> ----------------------------------------
>> Date: Thu, 14 Apr 2011 11:29:23 +0200
>> From: [hidden email]
>> To: [hidden email]
>> Subject: [R] Identify period length of time series automatically?
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hi
>>
>> I have 10.000 simulations for a sensitivity analysis. I have done a few
>> sensitivity analysis for different response variables already,
>> but now, as most of the simulations (if not all) show some cyclic
>> behaviour, see how the independent input parameter influence the
>> frequency of the cyclic changes and "how cyclic" they actually are.
>>
>> So effectively, I have 39 values, and I want to identify automatically
>> the frequency / period length of the series and a kind of a measure on
>> "how cyclic" the series is.

Hi Mike,

thanks for your answer - it confirms my fears ...

>
> Probably google "Digital Signal Processing" or Fourier transform.
> From this, you resolve your time series into sinusoids of various components
> and you can separate peaks in line spectra from background noise.
> Depending on what you consider to be "cyclic" the analysis details
> will vary. If you look at things like amplitude and frequncy modulation
> of one sine wave with another and various relationships between carrier and
> modulation frequency, you can get some ideas of what to look for in spectra.

That is what I thought as well. As I have no idea about fourier
analysis, could you give me a small example in R, which gives me the
frequencies of the resulting sin waves after a fourier transformation?
I only see large matrices as return values when using e.g. fft().

>
> Alternatively, you can try to define exactly what you mean by "cyclic"
> and maybe make a better transform that discriminates that from acyclic
> but offhand I would suggest FFT and various tests on the spectra.

the shape of the fluctuations can be quite different - so no common
pattern there.

>
>
> Just off hand I'm not sure that 39 points would be a lot to go on
> but you can simulate some examples in R quite easily if you know
> what the data looks like in various cases you think may exist.

Well - the data is over a year summed up data from daily data points, so
I could easily go to daily data, which would be 365*39. But that would
make the analysis probably more difficult, as I have seasonal
fluctuations, and fluctuations over several years (1, 2, 3, 4, ...?;
depending on the parameters used for the simulation).

Any ideas on how to do this in R?

I have the feeling, that the quesion id more difficult then I thought...

Rainer


>
>
>
>
>
>>
>> How can I do that automatically without individual checking? I do not
>> want to do an eyeball assessment for 10.000 time series....
>>
>> Thanks,
>>
>> Rainer
>>
>> - --
>> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
>> Biology, UCT), Dipl. Phys. (Germany)
>>
>> Centre of Excellence for Invasion Biology
>> Stellenbosch University
>> South Africa
>
>      


- --
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      [hidden email]

Skype:      RMkrug
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2mz5QACgkQoYgNqgF2egqZ8QCfZrtSmYczWo+Gq9NgY25mtP5Q
LHwAn3qaWKoo2wkc4pjTe9skZhcW7UL+
=4uTI
-----END PGP SIGNATURE-----

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Identify period length of time series automatically?

Mike Marchywka














----------------------------------------

> Date: Thu, 14 Apr 2011 12:42:28 +0200
> From: [hidden email]
> To: [hidden email]
> CC: [hidden email]
> Subject: Re: [R] Identify period length of time series automatically?
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 14/04/11 11:57, Mike Marchywka wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > ----------------------------------------
> >> Date: Thu, 14 Apr 2011 11:29:23 +0200
> >> From: [hidden email]
> >> To: [hidden email]
> >> Subject: [R] Identify period length of time series automatically?
> >>
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >> Hi
> >>
> >> I have 10.000 simulations for a sensitivity analysis. I have done a few
> >> sensitivity analysis for different response variables already,
> >> but now, as most of the simulations (if not all) show some cyclic
> >> behaviour, see how the independent input parameter influence the
> >> frequency of the cyclic changes and "how cyclic" they actually are.
> >>
> >> So effectively, I have 39 values, and I want to identify automatically
> >> the frequency / period length of the series and a kind of a measure on
> >> "how cyclic" the series is.
>
> Hi Mike,
>
> thanks for your answer - it confirms my fears ...

Well, if it is just that you wanted an alt to FFT I have not
provided one so it shouldn't confirm that fear as others could
exist. Indeed, if you have an idea of what you want you may
be able to create your own.  It wasn't clear in OP that you
had looked at FFT.

>
> >
> > Probably google "Digital Signal Processing" or Fourier transform.
> > From this, you resolve your time series into sinusoids of various components
> > and you can separate peaks in line spectra from background noise.
> > Depending on what you consider to be "cyclic" the analysis details
> > will vary. If you look at things like amplitude and frequncy modulation
> > of one sine wave with another and various relationships between carrier and
> > modulation frequency, you can get some ideas of what to look for in spectra.
>
> That is what I thought as well. As I have no idea about fourier
> analysis, could you give me a small example in R, which gives me the
> frequencies of the resulting sin waves after a fourier transformation?
> I only see large matrices as return values when using e.g. fft().
>
> >
> > Alternatively, you can try to define exactly what you mean by "cyclic"
> > and maybe make a better transform that discriminates that from acyclic
> > but offhand I would suggest FFT and various tests on the spectra.
>
> the shape of the fluctuations can be quite different - so no common
> pattern there.
>
> >
> >
> > Just off hand I'm not sure that 39 points would be a lot to go on
> > but you can simulate some examples in R quite easily if you know
> > what the data looks like in various cases you think may exist.
>
> Well - the data is over a year summed up data from daily data points, so
> I could easily go to daily data, which would be 365*39. But that would
> make the analysis probably more difficult, as I have seasonal
> fluctuations, and fluctuations over several years (1, 2, 3, 4, ...?;
> depending on the parameters used for the simulation).

plot(abs(fft(x)) as presumably your confusion is due to complex
result and spectrum is probably all you care about at first.
If that works, there are various R facilities for peak detection
that may be of use. There may be programs to fit FFT assuming
discrete + contiuum noise but I have not looked, there are apparently
things like that for fitting mass spectra.




>
> Any ideas on how to do this in R?
>
> I have the feeling, that the quesion id more difficult then I thought...
>
> Rainer
>
>
> >
> >
> >
> >
> >
> >>
> >> How can I do that automatically without individual checking? I do not
> >> want to do an eyeball assessment for 10.000 time series....
> >>
> >> Thanks,
> >>
> >> Rainer
> >>
> >> - --
> >> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
> >> Biology, UCT), Dipl. Phys. (Germany)
> >>
> >> Centre of Excellence for Invasion Biology
> >> Stellenbosch University
> >> South Africa
> >
> >
>
>
> - --
> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
> Biology, UCT), Dipl. Phys. (Germany)
>
> Centre of Excellence for Invasion Biology
> Stellenbosch University
> South Africa
>
> Tel : +33 - (0)9 53 10 27 44
> Cell: +33 - (0)6 85 62 59 98
> Fax : +33 - (0)9 58 10 27 44
>
> Fax (D): +49 - (0)3 21 21 25 22 44
>
> email: [hidden email]
>
> Skype: RMkrug
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk2mz5QACgkQoYgNqgF2egqZ8QCfZrtSmYczWo+Gq9NgY25mtP5Q
> LHwAn3qaWKoo2wkc4pjTe9skZhcW7UL+
> =4uTI
> -----END PGP SIGNATURE-----
     
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.