working with zoo time index ??

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

working with zoo time index ??

skan
Hello

Where could I find examples on how to work with the time index in a timeseries  or zoo series?

Let say I've got this series

DATA
1990-01-01 10:00:00   0.900
1990-01-01 10:01:00   0.910
1990-01-01 10:03:00   0.905
1990-01-01 10:04:00   0.905
1990-01-01 10:05:00   0.890

.......................

2000-12-31 20:00:00   0.992


How do I make simple calculations such as ... ?
Calculate the mean of the first data every day. (mapply, for loop, tapply ?)
Transform data to a table,  with dates in one axis and  times in the other.

thanks
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
On Tue, Jun 15, 2010 at 8:27 AM, skan <[hidden email]> wrote:

>
> Hello
>
> Where could I find examples on how to work with the time index in a
> timeseries  or zoo series?
>
> Let say I've got this series
>
> DATA
> 1990-01-01 10:00:00   0.900
> 1990-01-01 10:01:00   0.910
> 1990-01-01 10:03:00   0.905
> 1990-01-01 10:04:00   0.905
> 1990-01-01 10:05:00   0.890
>
> .......................
>
> 2000-12-31 20:00:00   0.992
>
>
> How do I make simple calculations such as ... ?
> Calculate the mean of the first data every day. (mapply, for loop, tapply ?)
> Transform data to a table,  with dates in one axis and  times in the other.
>

There are three vignettes that come with zoo.  vignette() lists their
names and vignette("zoo") displays the one called zoo (similarly for
the other two).  Also see the help files: ?zoo, ?read.zoo,
?aggregate.zoo
and note the examples at the bottom of the help files.
Also library(help = zoo) lists the help files available.

Lines <- "1990-01-01 10:00:00   0.900
1990-01-01 10:01:00   0.910
1990-01-01 10:03:00   0.905
1990-01-01 10:04:00   0.905
1990-01-01 10:05:00   0.890
1990-01-02 10:00:00   0.940
1990-01-02 10:01:00   0.990"
library(zoo)
library(chron)
z <- read.zoo(textConnection(Lines), index = 1:2, FUN = function(x)
as.chron(paste(x[,1], x[,2])))

# take first data value for each day and then take their mean
mean(aggregate(z, as.Date, head, 1))

# create data frame from z made up of dates, times and value
# dates and times are chron package functions.
# (If you use a different date and time class then it would be different.)
data.frame(dates = dates(time(z)), times = times(time(z)), value = coredata(z))

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

steven mosher
Hi Gabor,

 Not sure where to report this, but

Mac 10.5.8
R: 11.1

When you examine the zoo vignette and hit the back button, you get a hang.
I havent tested with other vignettes and cant imagine that is is specific to
yours
FWIW.

Did I mention that zoo is great. Thx for your work on it.

On Tue, Jun 15, 2010 at 6:30 AM, Gabor Grothendieck <[hidden email]
> wrote:

> On Tue, Jun 15, 2010 at 8:27 AM, skan <[hidden email]> wrote:
> >
> > Hello
> >
> > Where could I find examples on how to work with the time index in a
> > timeseries  or zoo series?
> >
> > Let say I've got this series
> >
> > DATA
> > 1990-01-01 10:00:00   0.900
> > 1990-01-01 10:01:00   0.910
> > 1990-01-01 10:03:00   0.905
> > 1990-01-01 10:04:00   0.905
> > 1990-01-01 10:05:00   0.890
> >
> > .......................
> >
> > 2000-12-31 20:00:00   0.992
> >
> >
> > How do I make simple calculations such as ... ?
> > Calculate the mean of the first data every day. (mapply, for loop, tapply
> ?)
> > Transform data to a table,  with dates in one axis and  times in the
> other.
> >
>
> There are three vignettes that come with zoo.  vignette() lists their
> names and vignette("zoo") displays the one called zoo (similarly for
> the other two).  Also see the help files: ?zoo, ?read.zoo,
> ?aggregate.zoo
> and note the examples at the bottom of the help files.
> Also library(help = zoo) lists the help files available.
>
> Lines <- "1990-01-01 10:00:00   0.900
> 1990-01-01 10:01:00   0.910
> 1990-01-01 10:03:00   0.905
> 1990-01-01 10:04:00   0.905
> 1990-01-01 10:05:00   0.890
> 1990-01-02 10:00:00   0.940
> 1990-01-02 10:01:00   0.990"
> library(zoo)
> library(chron)
> z <- read.zoo(textConnection(Lines), index = 1:2, FUN = function(x)
> as.chron(paste(x[,1], x[,2])))
>
> # take first data value for each day and then take their mean
> mean(aggregate(z, as.Date, head, 1))
>
> # create data frame from z made up of dates, times and value
> # dates and times are chron package functions.
> # (If you use a different date and time class then it would be different.)
> data.frame(dates = dates(time(z)), times = times(time(z)), value =
> coredata(z))
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
On Tue, Jun 15, 2010 at 1:54 PM, steven mosher <[hidden email]> wrote:

> Hi Gabor,
>  Not sure where to report this, but
> Mac 10.5.8
> R: 11.1
> When you examine the zoo vignette and hit the back button, you get a hang.
> I havent tested with other vignettes and cant imagine that is is specific to
> yours
> FWIW.
> Did I mention that zoo is great. Thx for your work on it.
>

If this is a problem in reading pdf files then its likely either a
problem with the pdf reader software itself or some Mac specific
problem that you could discuss on the r-sig-mac list.

You can also read the zoo vignettes online here:
http://cran.r-project.org/web/packages/zoo/index.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
Hi
thanks


Let say data are written like this:
1990-01-01 10:01:00 ,  0.910
1990-01-01 10:03:00 ,  0.905

Would it be ok to read it with theses lines or is better to use your way?

tmp <- read.table("demo2.txt", sep = ",")
z <- zoo(tmp[, 2], as.Date(as.chron(tmp[, 1]), format = "%Y-%m-%d %H:%M:%S"))

regards
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
I can't see where you check the date.
For example, if I want to save the "core" value everyday at 14:00:00
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
In reply to this post by skan
On Wed, Jun 16, 2010 at 7:58 AM, skan <[hidden email]> wrote:

>
> Hi
> thanks
>
>
> Let say data are written like this:
> 1990-01-01 10:01:00 ,  0.910
> 1990-01-01 10:03:00 ,  0.905
>
> Would it be ok to read it with theses lines or is better to use your way?
>
> tmp <- read.table("demo2.txt", sep = ",")
> z <- zoo(tmp[, 2], as.Date(as.chron(tmp[, 1]), format = "%Y-%m-%d
> %H:%M:%S"))

You want to avoid creating zoo objects that have duplicate times since
they can't be merged.  This will read the data in and at the same time
aggregate it by date using the first value among all values with the
same date.

Lines <- "1990-01-01 10:01:00 ,  0.910
1990-01-01 10:03:00 ,  0.905
1990-01-02 10:03:00 ,  0.895"

library(zoo)
z <- read.zoo(textConnection(Lines), sep = ",", FUN = as.Date,
        aggregate = function(x) head(x, 1))

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
On Wed, Jun 16, 2010 at 8:36 AM, Gabor Grothendieck
<[hidden email]> wrote:

> On Wed, Jun 16, 2010 at 7:58 AM, skan <[hidden email]> wrote:
>>
>> Hi
>> thanks
>>
>>
>> Let say data are written like this:
>> 1990-01-01 10:01:00 ,  0.910
>> 1990-01-01 10:03:00 ,  0.905
>>
>> Would it be ok to read it with theses lines or is better to use your way?
>>
>> tmp <- read.table("demo2.txt", sep = ",")
>> z <- zoo(tmp[, 2], as.Date(as.chron(tmp[, 1]), format = "%Y-%m-%d
>> %H:%M:%S"))
>
> You want to avoid creating zoo objects that have duplicate times since
> they can't be merged.  This will read the data in and at the same time
> aggregate it by date using the first value among all values with the
> same date.
>
> Lines <- "1990-01-01 10:01:00 ,  0.910
> 1990-01-01 10:03:00 ,  0.905
> 1990-01-02 10:03:00 ,  0.895"
>
> library(zoo)
> z <- read.zoo(textConnection(Lines), sep = ",", FUN = as.Date,
>        aggregate = function(x) head(x, 1))
>

This can be reduced slightly to the following as it defaults to Date here:

z <- read.zoo(textConnection(Lines), sep = ",", aggregate =
function(x) head(x, 1))

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
This post was updated on .
Hi
I'll ask it in a different way...

I have all this in a file.txt

1990-01-01 10:00:00 ,  0.900          #  element 1
1990-01-01 10:01:00 ,  0.910          #  element 2
1990-01-01 10:03:00 ,  0.905          #  element 3
1990-01-01 10:04:00 ,  0.905          #  element 4
1990-01-01 10:05:00 ,  0.890          #  element 5
..................................................................
2000-12-30 20:00:00 ,  11.233        # element 3323232

How do I loop through the index? first element 1, then element 2, then the third...
How do I extract the day or the hour or the minutes from an element from the index(element n)?
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
This post was updated on .
I said taking the first element everyday, but that was just an example, I could need one every 2 hours or something more complicated such as one every hour if the former one was not null.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
In reply to this post by skan
On Wed, Jun 16, 2010 at 12:40 PM, skan <[hidden email]> wrote:

>
> Hi
> I'll ask in a different way...
>
> I have all this in a file.txt
>
> 1990-01-01 10:00:00 ,  0.900          #  element 1
> 1990-01-01 10:01:00 ,  0.910          #  element 2
> 1990-01-01 10:03:00 ,  0.905          #  element 3
> 1990-01-01 10:04:00 ,  0.905          #  element 4
> 1990-01-01 10:05:00 ,  0.890          #  element 5
> ..................................................................
> 2000-12-30 20:00:00 ,  11.233        # element 3323232
>
> How do I loop through the index? first element 1, then element 2, then the
> third...

That is not the R way.  Why do you want to loop through it?  What do
you want to do?  The R way is the whole object approach.

> How do I extract the day or the hour or the minutes from an element from the
> index(element n)?

See R News 4/1 for info on dates and times.  See the three vignettes
in zoo and the help files for info on zoo. There are tons of examples
there.

Here is some code. The loops are not recommended but are only there
since you asked.

Lines <- "1990-01-01 10:00:00 ,  0.900          #  element 1
1990-01-01 10:01:00 ,  0.910          #  element 2
1990-01-01 10:03:00 ,  0.905          #  element 3
1990-01-01 10:04:00 ,  0.905          #  element 4
1990-01-01 10:05:00 ,  0.890          #  element 5
2000-12-30 20:00:00 ,  11.233        # element 3323232"

library(zoo)
library(chron)
z <- read.zoo(textConnection(Lines), sep = ",", FUN = as.chron)
z

for(i in seq_along(z)) print(time(z)[i])
for(i in seq_along(z)) print(coredata(z)[i])

hours(time(z))
minutes(time(z))

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
In reply to this post by skan
On Wed, Jun 16, 2010 at 1:10 PM, skan <[hidden email]> wrote:
>
> I said taking the first element everyday, but that was just an example, I
> could need one every 2 hours or something more complicated such as one every
> hour if the former one was non null.

Lines <- "1990-01-01 10:00:00 ,  0.900          #  element 1
1990-01-01 10:01:00 ,  0.910          #  element 2
1990-01-01 10:03:00 ,  0.905          #  element 3
1990-01-01 10:04:00 ,  0.905          #  element 4
1990-01-01 10:05:00 ,  0.890          #  element 5
2000-12-30 20:00:00 ,  11.233        # element 3323232"

library(zoo)
z <- read.zoo(textConnection(Lines), sep = ",", tz = "")

# take mean of every 2 hour segment
aggregate(z, as.POSIXct(cut(time(z), "2 hours", include = TRUE)), mean)

For more examples, see:
?aggregate.zoo

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
On Wed, Jun 16, 2010 at 4:55 PM, Gabor Grothendieck
<[hidden email]> wrote:

> On Wed, Jun 16, 2010 at 1:10 PM, skan <[hidden email]> wrote:
>>
>> I said taking the first element everyday, but that was just an example, I
>> could need one every 2 hours or something more complicated such as one every
>> hour if the former one was non null.
>
> Lines <- "1990-01-01 10:00:00 ,  0.900          #  element 1
> 1990-01-01 10:01:00 ,  0.910          #  element 2
> 1990-01-01 10:03:00 ,  0.905          #  element 3
> 1990-01-01 10:04:00 ,  0.905          #  element 4
> 1990-01-01 10:05:00 ,  0.890          #  element 5
> 2000-12-30 20:00:00 ,  11.233        # element 3323232"
>
> library(zoo)
> z <- read.zoo(textConnection(Lines), sep = ",", tz = "")
>
> # take mean of every 2 hour segment
> aggregate(z, as.POSIXct(cut(time(z), "2 hours", include = TRUE)), mean)
>
> For more examples, see:
> ?aggregate.zoo
>

And here it is using chron:


Lines <- "1990-01-01 10:00:00 ,  0.900          #  element 1
1990-01-01 10:01:00 ,  0.910          #  element 2
1990-01-01 10:03:00 ,  0.905          #  element 3
1990-01-01 10:04:00 ,  0.905          #  element 4
1990-01-01 10:05:00 ,  0.890          #  element 5
2000-12-30 20:00:00 ,  11.233        # element 3323232"
library(zoo)
library(chron)
z <- read.zoo(textConnection(Lines), sep = ",", FUN = as.chron)
# take mean of every 2 hour segment
aggregate(z, trunc(time(z), "02:00:00"), mean)

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
thanks
How can I do it without using  "aggregate"?
In other languages they use commands like Time[i]  or  Date[i]<>Date[i-1]   were i is the cell
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
OK,
I've seen now your reply now

thanks very much
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
Hi again.


I have several files with data like above. Each file has different periods.

My idea is to use zoo in order to do this...
Converting all data to same period, the smaller one, 5 minutes.
Whenever a datum doesn't exist copy the last one. (carry forward)
Add data of every 5 minutes getting a new series.
With the new series....
run a loop:
every day I save the first data
I substract every datum within this day from the saved data for tihs day
And I check if that substraction accomplishes some test, for example equals a number.
I this test is not fulfilled I continue to the end of the day. Then I store the value of the subratcion last an I start with the next day.
If it's fulffiled I store the value of the substraction that fulfilled the test and I jump directly to the next day.

Can you do all this with "aggregate" ?
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
What if I use something like
myvalue = coredata[ index[x] == as.Date("2009-03-01") ]
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
In reply to this post by skan
On Thu, Jun 17, 2010 at 7:38 AM, skan <[hidden email]> wrote:

>
> Hi again.
>
>
> I have several files with data like above. Each file has different periods.
>
> My idea is to use zoo in order to do this...
> Converting all data to same period, the smaller one, 5 minutes.
> Whenever a datum doesn't exist copy the last one. (carry forward)
> Add data of every 5 minutes getting a new series.
> With the new series....
> run a loop:
> every day I save the first data
> I substract every datum within this day from the saved data for tihs day
> And I check if that substraction accomplishes some test, for example equals
> a number.
> I this test is not fulfilled I continue to the end of the day. Then I store
> the value of the subratcion last an I start with the next day.
> If it's fulffiled I store the value of the substraction that fulfilled the
> test and I jump directly to the next day.

As already suggested read the three vignettes that come with zoo and
also read ?aggregate.zoo and look at the examples in that help file.
Also look at other help files as needed and the answers I have already
given.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

Gabor Grothendieck
In reply to this post by skan
On Thu, Jun 17, 2010 at 9:40 AM, skan <[hidden email]> wrote:
>
> What if I use something like
> myvalue = coredata[ index[x] == as.Date("2009-03-01") ]

See ?window.zoo

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: working with zoo time index ??

skan
I've read all these documents and some other.