R › R help

DateTime Math in R - POSIXct

Classic

List

Threaded

8 messages Options

Galen Moore

DateTime Math in R - POSIXct

Greetings -

I'm battling POSIXct, as per the code below. My input is actually an XL
file, but the weird results below correctly model what I am seeing in my
program.

Before I punt and use lubridate or timeDate, could anyone please help me
understand why POSIXct forces my variable back to GMT?

I suspect that I'm not properly coding the tzone value, but it does not
throw an error as-is.

> tstamp <- "2011-05-22 11:45:00 MDT"

> mode(tstamp)

[1] "character"

>

> dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")

> mode(dateP)

[1] "numeric"

> dateP

[1] "2011-05-22 11:45:00 MDT"

>

> dateN <- as.numeric(dateP)

> dateN

[1] 1306086300

>

> dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")

> dateP2

[1] "2011-05-22 18:45:00 MDT"

Many thanks.

Galen Moore

[[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

David Winsemius

Re: DateTime Math in R - POSIXct

On May 30, 2011, at 10:20 PM, Galen Moore wrote:

> Greetings -
>
>
>
> I'm battling POSIXct, as per the code below. My input is actually
> an XL
> file, but the weird results below correctly model what I am seeing
> in my
> program.
>
>
>
> Before I punt and use lubridate or timeDate, could anyone please
> help me
> understand why POSIXct forces my variable back to GMT?
>
>
>
> I suspect that I'm not properly coding the tzone value, but it does
> not
> throw an error as-is.
>
>
>
>
>
>> tstamp <- "2011-05-22 11:45:00 MDT"
>
>> mode(tstamp)
>
> [1] "character"
>
>>
>
>> dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")
>
>> mode(dateP)
>
> [1] "numeric"
>
>> dateP
>
> [1] "2011-05-22 11:45:00 MDT"
>
>>
>
>> dateN <- as.numeric(dateP)
>
>> dateN
>
> [1] 1306086300
>

Bill.Venables

Re: DateTime Math in R - POSIXct

In reply to this post by Galen Moore

Perhaps because the timezone is specified as a character string and not a date-time object complete with timezone.

>From the help filr for as.POSIXct.numeric:

"origin: a date-time object, or something which can be coerced by as.POSIXct(tz="GMT") to such an object."

Note the coercion.

Bill Venables.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Galen Moore
Sent: Tuesday, 31 May 2011 12:20 PM
To: [hidden email]
Subject: [R] DateTime Math in R - POSIXct

Greetings -

I'm battling POSIXct, as per the code below. My input is actually an XL
file, but the weird results below correctly model what I am seeing in my
program.

Before I punt and use lubridate or timeDate, could anyone please help me
understand why POSIXct forces my variable back to GMT?

I suspect that I'm not properly coding the tzone value, but it does not
throw an error as-is.

> tstamp <- "2011-05-22 11:45:00 MDT"

> mode(tstamp)

[1] "character"

>

> dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")

> mode(dateP)

[1] "numeric"

> dateP

[1] "2011-05-22 11:45:00 MDT"

>

> dateN <- as.numeric(dateP)

> dateN

[1] 1306086300

>

> dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")

> dateP2

[1] "2011-05-22 18:45:00 MDT"

Many thanks.

Galen Moore

[[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Galen Moore

Re: DateTime Math in R - POSIXct

I think that has something to do with it. I'll keep chewing on things and
see what I can make work.

I've reached a point where I get the right answer for the wrong reasons....
;-)

Many thanks,

Galen

-----Original Message-----
From: [hidden email] [mailto:[hidden email]]
Sent: Monday, May 30, 2011 21:22
To: [hidden email]; [hidden email]
Subject: RE: [R] DateTime Math in R - POSIXct

Perhaps because the timezone is specified as a character string and not a
date-time object complete with timezone.

>From the help filr for as.POSIXct.numeric:

"origin: a date-time object, or something which can be coerced by
as.POSIXct(tz="GMT") to such an object."

Note the coercion.

Bill Venables.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Galen Moore
Sent: Tuesday, 31 May 2011 12:20 PM
To: [hidden email]
Subject: [R] DateTime Math in R - POSIXct

Greetings -

I'm battling POSIXct, as per the code below. My input is actually an XL
file, but the weird results below correctly model what I am seeing in my
program.

Before I punt and use lubridate or timeDate, could anyone please help me
understand why POSIXct forces my variable back to GMT?

I suspect that I'm not properly coding the tzone value, but it does not
throw an error as-is.

> tstamp <- "2011-05-22 11:45:00 MDT"

> mode(tstamp)

[1] "character"

>

> dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")

> mode(dateP)

[1] "numeric"

> dateP

[1] "2011-05-22 11:45:00 MDT"

>

> dateN <- as.numeric(dateP)

> dateN

[1] 1306086300

>

> dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")

> dateP2

[1] "2011-05-22 18:45:00 MDT"

Many thanks.

Galen Moore

[[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Galen Moore

Re: DateTime Math in R - POSIXct

In reply to this post by Galen Moore

Thank you, David.

I am not subtracting the seconds to convert POSIXct's GMT to MDT, and don't
understand why I should need to.

Any hints, however, as to why
dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")
returns a date in the correct tz in my first instance below, yet

dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")
decides to revert to GMT in my second instance below?

Thanks,

Galen

-----Original Message-----
From: David Winsemius [mailto:[hidden email]]
Sent: Monday, May 30, 2011 20:31
To: [hidden email]
Cc: [hidden email]
Subject: Re: [R] DateTime Math in R - POSIXct

On May 30, 2011, at 10:20 PM, Galen Moore wrote:

> Greetings -
>
>
>
> I'm battling POSIXct, as per the code below. My input is actually an
> XL file, but the weird results below correctly model what I am seeing
> in my program.
>
>
>
> Before I punt and use lubridate or timeDate, could anyone please help
> me understand why POSIXct forces my variable back to GMT?
>
>
>
> I suspect that I'm not properly coding the tzone value, but it does
> not throw an error as-is.
>
>
>
>
>
>> tstamp <- "2011-05-22 11:45:00 MDT"
>
>> mode(tstamp)
>
> [1] "character"
>
>>
>
>> dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")
>
>> mode(dateP)
>
> [1] "numeric"
>
>> dateP
>
> [1] "2011-05-22 11:45:00 MDT"
>
>>
>
>> dateN <- as.numeric(dateP)
>
>> dateN
>
> [1] 1306086300
>

Jeff Newmiller

Re: DateTime Math in R - POSIXct

MDT is not a valid timezone specification... it is falling back on your default timezone in your "working" case.

a) learn what valid timezone specifications are on your operating system. I suspect you are using Windows (hint: we should not have to guess), so look for the timezone directory in your R installation directory. I would guess that you want "America/Denver". Note that MST and MST7MDT are also possibilities , but they are generally deprecated as being too imprecise in the global context (who is to say "MDT" doesn't mean "Mongolian Daylight Time").

b) I strongly recommend using the "default" timezone or overriding it with Sys.setenv(TZ= your timezone spec) rather than using the tz arguments. The frustration factor is much lower.

c) if you properly import all of your time values into POSIXct, their internal representations will always be referenced to UTC (GMT). It is only on input and output (such as printing to the console) or conversion to POSIXlt that the timezone is considered.
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<[hidden email]> Basics: ##.#. ##.#. Live Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---------------------------------------------------------------------------
Sent from my phone. Please excuse my brevity.

Galen Moore <[hidden email]> wrote:

Thank you, David.

I am not subtracting the seconds to convert POSIXct's GMT to MDT, and don't
understand why I should need to.

Any hints, however, as to why
dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")
returns a date in the correct tz in my first instance below, yet

dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")
decides to revert to GMT in my second instance below?

Thanks,

Galen

-----Original Message-----
From: David Winsemius [mailto:[hidden email]]
Sent: Monday, May 30, 2011 20:31
To: [hidden email]
Cc: [hidden email]
Subject: Re: [R] DateTime Math in R - POSIXct

On May 30, 2011, at 10:20 PM, Galen Moore wrote:

So now the internal representation is referenced to GMT

>>
>
>> dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")
>
>> dateP2

And you are telling the function to add as many hours offset as needed for
the difference between GMT and MD T.....

--
David.
>
> [1] "2011-05-22 18:45:00 MDT"
>

David Winsemius, MD
West Hartford, CT

_____________________________________________

[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Galen Moore

Re: DateTime Math in R - POSIXct

Many thank, Jeff –

a) I am indeed on Windoze, and found the time zone directory under: C:\Program Files\R\R-2.13.0\share\zoneinfo
b) I applied Sys.setenv(TZ="America/Denver") to my real program, and the numerical and POSIXct output representations of my date/times now work, thanks! I've avoided converting back to numeric from POSIXct in my real program. I left my calc'd variables in numeric mode so as to avoid POSIXct surprises.
c) Oddly, however, my dummy/demo code below still shows some screwy behavior, even with Sys.setenv(TZ="America/Denver") applied. I'm guessing that the apparent reversion to GMT occurs in my statement " dateN <- as.numeric(dateP)" below is merely the internal representation to which you refer. I don't understand why POSIXct slaps another hour on dateN on conversion to dateP, nor why R generates the "MDT" timezone value (not listed and not valid tz designation).

Thanks again for your help, and grateful for any further hints. I'd really like to be not confused by this approach to datetime arithmetic in R.

- Galen

> Sys.setenv(TZ="America/Denver")
>
> tstamp <- "2011-05-22 11:45:00"
> mode(tstamp)
[1] "character"
>
> dateP <- as.POSIXct(tstamp, origin="1970-01-01")
> mode(dateP)
[1] "numeric"
> dateP
[1] "2011-05-22 11:45:00 MDT"
> # Denver is currently on MDT, and note that R generates the "MDT" designation,
> # even though it is not listed in C:\Program Files\R\R-2.13.0\share\zoneinfo
>
> dateN <- as.numeric(dateP)
> dateN
[1] 1306086300
> # 1306086300 returns "5/22/2011 17:45:00" when converted manually
>
> dateP2 <- as.POSIXct(dateN, origin="1970-01-01")
> dateP2
[1] "2011-05-22 18:45:00 MDT"
> # Where does the extra hour come from?

From: Jeff Newmiller [mailto:[hidden email]]
Sent: Tuesday, May 31, 2011 00:43
To: [hidden email]; 'David Winsemius'
Cc: [hidden email]
Subject: Re: [R] DateTime Math in R - POSIXct

MDT is not a valid timezone specification... it is falling back on your default timezone in your "working" case.

a) learn what valid timezone specifications are on your operating system. I suspect you are using Windows (hint: we should not have to guess), so look for the timezone directory in your R installation directory. I would guess that you want "America/Denver". Note that MST and MST7MDT are also possibilities , but they are generally deprecated as being too imprecise in the global context (who is to say "MDT" doesn't mean "Mongolian Daylight Time").

b) I strongly recommend using the "default" timezone or overriding it with Sys.setenv(TZ= your timezone spec) rather than using the tz arguments. The frustration factor is much lower.

c) if you properly import all of your time values into POSIXct, their internal representations will always be referenced to UTC (GMT). It is only on input and output (such as printing to the console) or conversion to POSIXlt that the timezone is considered.
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<[hidden email]> Basics: ##.#. ##.#. Live Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---------------------------------------------------------------------------
Sent from my phone. Please excuse my brevity.
Galen Moore <[hidden email]> wrote:
Thank you, David.

I am not subtracting the seconds to convert POSIXct's GMT to MDT, and don't
understand why I should need to.

Any hints, however, as to why
dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")
returns a date in the correct tz in my first instance below, yet

dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")
decides to revert to GMT in my second instance below?

Thanks,

Galen

-----Original Message-----
From: David Winsemius [mailto:[hidden email]]
Sent: Monday, May 30, 2011 20:31
To: [hidden email]
Cc: [hidden email]
Subject: Re: [R] DateTime Math in R - POSIXct

On May 30, 2011, at 10:20 PM, Galen Moore wrote:

> Greetings -
>
>
>
> I'm battling
g
POSIXct, as per the code below. My input is actually an

> XL file, but the weird results below correctly model what I am seeing
> in my program.
>
>
>
> Before I punt and use lubridate or timeDate, could anyone please help
> me understand why POSIXct forces my variable back to GMT?
>
>
>
> I suspect that I'm not properly coding the tzone value, but it does
> not throw an error as-is.
>
>
>
>
>
>> tstamp <- "2011-05-22 11:45:00 MDT"
>
>> mode(tstamp)
>
> [1] "character"
>
>>
>
>> dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")
>
>> mode(dateP)
>
> [1] "numeric"
>
>> dateP
>
> [1] "2011-05-22 11:45:00 MDT"
>
>>
>
>> dateN <-

as.numeric(dateP)
>
>> dateN
>
> [1] 1306086300
>

So now the internal representation is referenced to GMT

>>
>
>> dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")
>
>> dateP2

And you are telling the function to add as many hours offset as needed for
the difference between GMT and MD T.....

--
David.
>
> [1] "2011-05-22 18:45:00 MDT"
>

David Winsemius, MD
West Hartford, CT
________________________________________

[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Jeff Newmiller

Re: DateTime Math in R - POSIXct

Although the documentation for as.POSIXct indicates that an origin string value will be converted using tz="GMT", it is actually converted using tz="" (=default timezone) and the result acquires that timezone rather than the tz argument timezone.

As Dr. Ripley alluded, you need to use an origin POSIXt object rather than a string to get the behavior you want, e.g.

myorigin <- 0 # POSIX origin regardless of timezone
class( myorigin ) <- c( "POSIXt", "POSIXct" )
dateP2 <- as.POSIXct( dateN, origin=myorigin )

I think there should be an easier way to do this, but in the long run you are better off avoiding converting back and forth between numeric and POSIX times anyway.
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<[hidden email]> Basics: ##.#. ##.#. Live Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---------------------------------------------------------------------------
Sent from my phone. Please excuse my brevity.

Galen Moore <[hidden email]> wrote:

Many thank, Jeff â

a) I am indeed on Windoze, and found the time zone directory under: C:\Program Files\R\R-2.13.0\share\zoneinfo
b) I applied Sys.setenv(TZ="America/Denver") to my real program, and the numerical and POSIXct output representations of my date/times now work, thanks! I've avoided converting back to numeric from POSIXct in my real program. I left my calc'd variables in numeric mode so as to avoid POSIXct surprises.
c) Oddly, however, my dummy/demo code below still shows some screwy behavior, even with Sys.setenv(TZ="America/Denver") applied. I'm guessing that the apparent reversion to GMT occurs in my statement " dateN <- as.numeric(dateP)" below is merely the internal representation to which you refer. I don't understand why POSIXct slaps another hour on dateN on conversion to dateP, nor why R generates the "MDT" timezone value (not listed and not valid tz designation).

Thanks again for your help, and grateful for any further hints. I'd really like to be not confused by this approach to datetime arithmetic in R.

- Galen

> Sys.setenv(TZ="America/Denver")
>
> tstamp <- "2011-05-22 11:45:00"
> mode(tstamp)
[1] "character"
>
> dateP <- as.POSIXct(tstamp, origin="1970-01-01")
> mode(dateP)
[1] "numeric"
> dateP
[1] "2011-05-22 11:45:00 MDT"
> # Denver is currently on MDT, and note that R generates the "MDT" designation,
> # even though it is not listed in C:\Program Files\R\R-2.13.0\share\zoneinfo
>
> dateN <- as.numeric(dateP)
> dateN
[1] 1306086300
> # 1306086300 returns "5/22/2011 17:45:00" when converted manually
>
> dateP2 <- as.POSIXct(dateN, origin="1970-01-01")
> dateP2
[1] "2011-05-22 18:45:00 MDT"
> # Where does the extra hour come from?

From: Jeff Newmiller [mailto:[hidden email]]
Sent: Tuesday, May 31, 2011 00:43
To: [hidden email]; 'David Winsemius'
Cc: [hidden email]
Subject: Re: [R] DateTime Math in R - POSIXct

MDT is not a valid timezone specification... it is falling back on your default timezone in your "working" case.

a) learn what valid timezone specifications are on your operating system. I suspect you are using Windows (hint: we should not have to guess), so look for the timezone directory in your R installation directory. I would guess that you want "America/Denver". Note that MST and MST7MDT are also possibilities , but they are generally deprecated as being too imprecise in the global context (who is to say "MDT" doesn't mean "Mongolian Daylight Time").

b) I strongly recommend using the "default" timezone or overriding it with Sys.setenv(TZ= your timezone spec) rather than using the tz arguments. The frustration factor is much lower.

c) if you properly import all of your time values into POSIXct, their internal representations will always be referenced to UTC (GMT). It is only on input and output (such as printing to the console) or conversion to POSIXlt that the timezone is considered.
_____________________________________________

Jeff Newmiller The ..... ..... Go Live...
DCN:<[hidden email]> Basics: ##.#. ##.#. Live Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
_____________________________________________

Sent from my phone. Please excuse my brevity.
Galen Moore <[hidden email]> wrote:
Thank you, David.

I am not subtracting the seconds to convert POSIXct's GMT to MDT, and don't
understand why I should need to.

Any hints, however, as to why
dateP <- as.POSIXct(tstamp, origin="1970-01-01", tzone="MDT")
returns a date in the correct tz in my first instance below, yet

dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")
decides to revert to GMT in my second instance below?

Thanks,

Galen

-----Original Message-----
From: David Winsemius [mailto:[hidden email]]
Sent: Monday, May 30, 2011 20:31
To: [hidden email]
Cc: [hidden email]
Subject: Re: [R] DateTime Math in R - POSIXct

On May 30, 2011, at 10:20 PM, Galen Moore wrote:

> Greetings -
>
>
>
> I'm battling
g
POSIXct, as per the code below. My input is actually an

as.numeric(dateP)
>
>> dateN
>
> [1] 1306086300
>

So now the internal representation is referenced to GMT

>>
>
>> dateP2 <- as.POSIXct(dateN, origin="1970-01-01", tzone="MDT")
>
>> dateP2

And you are telling the function to add as many hours offset as needed for
the difference between GMT and MD T.....

--
David.
>
> [1] "2011-05-22 18:45:00 MDT"
>

David Winsemius, MD
West Hartford, CT
_____________________________________________

[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.