Read in data table, change columns from factors

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Read in data table, change columns from factors

Rich Shepard
   A set of data files have this format:

date,time,elev
1988-10-01,00:30,87.6849
1988-10-01,01:00,87.6849
1988-10-01,01:30,87.6849
1988-10-01,02:00,87.6879
1988-10-01,02:30,87.6879
1988-10-01,03:00,87.691
1988-10-01,03:30,87.694

Importing it with this command:

allyears <- read.table('allyears.dat', header = T, sep = ',')

produces this structure:

str(allyears)
'data.frame': 402414 obs. of  3 variables:
  $ date: Factor w/ 10230 levels "'data'","'date'",..:
  $ time: Factor w/ 1441 levels "'time'","00:00",..:
  $ elev: Factor w/ 4494 levels "'elev'","-3762938880000000369098752",..:

   Applying,
allyears$date <- as.Date(as.character(allyears$date))
changes the structure to
str(allyears)
'data.frame': 402414 obs. of  3 variables:
  $ date: Date, format: "1988-10-01" "1988-10-01" ...
  $ time: Factor w/ 1441 levels "'time'","00:00",..:
  $ elev: Factor w/ 4494 levels "'elev'","-3762938880000000369098752",..:

   I've not found the proper syntax to change time to a POSIXct time format
nor the elev to a numeric format.

allyears$time <- as.POSIXct(as.character(allyears$time, format=%H:%M))
Error: unexpected SPECIAL in "allyears$time <- as.POSIXct(as.character(allyears$time, format=%H:%"

and

allyears$elev <- as.Numeric(allyears$elev)
Error in as.Numeric(allyears$elev) : could not find function "as.Numeric"

   I've read ?read.table and looked on the web but I'm not finding how to
properly change the time and elev factors to H:M and fractional feet. A
pointer to a resource is much appreciated.

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

David Winsemius

> On Jul 18, 2018, at 2:50 PM, Rich Shepard <[hidden email]> wrote:
>
>  A set of data files have this format:
>
> date,time,elev
> 1988-10-01,00:30,87.6849
> 1988-10-01,01:00,87.6849
> 1988-10-01,01:30,87.6849
> 1988-10-01,02:00,87.6879
> 1988-10-01,02:30,87.6879
> 1988-10-01,03:00,87.691
> 1988-10-01,03:30,87.694
>
> Importing it with this command:
>
> allyears <- read.table('allyears.dat', header = T, sep = ',')
>
> produces this structure:
>
> str(allyears)
> 'data.frame': 402414 obs. of  3 variables:
> $ date: Factor w/ 10230 levels "'data'","'date'",..:
> $ time: Factor w/ 1441 levels "'time'","00:00",..:
> $ elev: Factor w/ 4494 levels "'elev'","-3762938880000000369098752",..:
>
>  Applying,
> allyears$date <- as.Date(as.character(allyears$date))
> changes the structure to
> str(allyears)
> 'data.frame': 402414 obs. of  3 variables:
> $ date: Date, format: "1988-10-01" "1988-10-01" ...
> $ time: Factor w/ 1441 levels "'time'","00:00",..:
> $ elev: Factor w/ 4494 levels "'elev'","-3762938880000000369098752",..:
>
>  I've not found the proper syntax to change time to a POSIXct time format
> nor the elev to a numeric format.

I would not destroy the possibility of using the original values:

> allyears$myDate <- as.Date(as.character(allyears$date))
> allyears$myTime <- as.POSIXct(paste(allyears$date, allyears$time))
> allyears
        date  time    elev     myDate              myTime
1 1988-10-01 00:30 87.6849 1988-10-01 1988-10-01 00:30:00
2 1988-10-01 01:00 87.6849 1988-10-01 1988-10-01 01:00:00
3 1988-10-01 01:30 87.6849 1988-10-01 1988-10-01 01:30:00
4 1988-10-01 02:00 87.6879 1988-10-01 1988-10-01 02:00:00
5 1988-10-01 02:30 87.6879 1988-10-01 1988-10-01 02:30:00
6 1988-10-01 03:00  87.691 1988-10-01 1988-10-01 03:00:00
7 1988-10-01 03:30 87.694t 1988-10-01 1988-10-01 03:30:00

>


> allyears$time <- as.POSIXct(as.character(allyears$time, format=%H:%M))
> Error: unexpected SPECIAL in "allyears$time <- as.POSIXct(as.character(allyears$time, format=%H:%"
>
> and
>
> allyears$elev <- as.Numeric(allyears$elev)
> Error in as.Numeric(allyears$elev) : could not find function "as.Numeric"

It's spelled `as.numeric`, but I'm having difficulty thinking about what sort of elevation (or would that be "depth") would be be measured by "-3762938880000000369098752".


>
>  I've read ?read.table and looked on the web but I'm not finding how to
> properly change the time and elev factors to H:M and fractional feet. A
> pointer to a resource is much appreciated.
>
> Rich
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.'   -Gehm's Corollary to Clarke's Third Law

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

Rich Shepard
On Wed, 18 Jul 2018, David Winsemius wrote:

> I would not destroy the possibility of using the original values:

David,

   What are the benefits of keeping date and time as factors?

>> allyears$myDate <- as.Date(as.character(allyears$date))
>> allyears$myTime <- as.POSIXct(paste(allyears$date, allyears$time))

   The latter command is not working on the full (402415 rows in the allyears
data set):

allyears$myTime <- as.POSIXct(paste(allyears$date, allyears$time))
Error in as.POSIXlt.character(x, tz, ...) :
   character string is not in a standard unambiguous format

> It's spelled `as.numeric`,

   Ah, I missed that. Thank you.

> ... but I'm having difficulty thinking about what sort of elevation (or would
> that be "depth") would be be measured by "-3762938880000000369098752".

   I also was curious about that figure and grep doesn't find it in the
original data file so I've no idea where R-3.5.0 came up with it.

   I'll read more about the POSIX datetime functions.

Thanks again,

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

David Winsemius

> On Jul 18, 2018, at 4:07 PM, Rich Shepard <[hidden email]> wrote:
>
> On Wed, 18 Jul 2018, David Winsemius wrote:
>
>> I would not destroy the possibility of using the original values:
>
> David,
>
>  What are the benefits of keeping date and time as factors?
>
>>> allyears$myDate <- as.Date(as.character(allyears$date))
>>> allyears$myTime <- as.POSIXct(paste(allyears$date, allyears$time))


It's not so much as factors but rather in a form that paste() will coerce to character so you cna get the automatic format

>
>  The latter command is not working on the full (402415 rows in the allyears
> data set):
>
> allyears$myTime <- as.POSIXct(paste(allyears$date, allyears$time))
> Error in as.POSIXlt.character(x, tz, ...) :
>  character string is not in a standard unambiguous format

Maybe you need to add a format string. It might force some of your pasted date+time values to NA but at least you would be able to identify the original values and perhaps fix errors.

You really should include a large snapshot of data that will allow reproducibility.


>
>> It's spelled `as.numeric`,
>
>  Ah, I missed that. Thank you.
>
>> ... but I'm having difficulty thinking about what sort of elevation (or would
>> that be "depth") would be be measured by "-3762938880000000369098752".
>
>  I also was curious about that figure and grep doesn't find it in the
> original data file so I've no idea where R-3.5.0 came up with it.
>
>  I'll read more about the POSIX datetime functions.
>
> Thanks again,
>
> Rich
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.'   -Gehm's Corollary to Clarke's Third Law

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

Rich Shepard
On Wed, 18 Jul 2018, David Winsemius wrote:

> It's not so much as factors but rather in a form that paste() will coerce
> to character so you cna get the automatic format

David,

   Now I understand.

> Maybe you need to add a format string. It might force some of your pasted
> date+time values to NA but at least you would be able to identify the
> original values and perhaps fix errors.

   Thinking about the data I realize that rather than separate date and time
columns what's needed is a datetime string. I'll add the format string in
the morning and work on this.

> You really should include a large snapshot of data that will allow
> reproducibility.

   I'll certainly do this. With hourly and half-hourly data from 1989-June
2018 (but missing all of 1992) there are more than 400K rows in the raw data
file. If you would suggest how many would be an acceptably large number I'll
be happy to put that on a 'cloud' sharing site and provide the URL to it.

Best regards,

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

Rich Shepard
On Wed, 18 Jul 2018, Rich Shepard wrote:

> If you would suggest how many would be an acceptably large number I'll be
> happy to put that on a 'cloud' sharing site and provide the URL to it.

   I put a zipped data file at this URL:
<http://www.fileconvoy.com/dfl.php?id=g61a366bb8947de43100009863935141c96f82092d4>

   It will stay there for 5 days.

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

Rich Shepard
On Thu, 19 Jul 2018, Rich Shepard wrote:

>  I put a zipped data file at this URL:
> <http://www.fileconvoy.com/dfl.php?id=g61a366bb8947de43100009863935141c96f82092d4>

   Since then I reformatted the file to two fields: date-time and elevation.
If anyone wants a copy send me a message off the list and I'll respond with
the modified file attached.

   Because time zone doesn't matter for a single location I'm looking at how
to use chron(). I've downloaded the PDF from CRAN and am trying to apply it
correctly so the dataframe contains two columns: date-time and elevation.
Will probably be asking for help in correctly using chron().

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

Rich Shepard
On Thu, 19 Jul 2018, Rich Shepard wrote:

> Since then I reformatted the file to two fields: date-time and elevation.
> If anyone wants a copy send me a message off the list and I'll respond with
> the modified file attached.

   This is a mistake. The file needs commas separating each field.

   I have the date and elev columns converted from factors to date and
numeric, respectively, but still have not learned how to convert the time.

   The source data file (head):
date,time,elev
2017-10-01,00:00,290.298
2017-10-01,00:30,290.301
2017-10-01,01:00,290.304
2017-10-01,01:30,290.295
2017-10-01,02:00,290.292
2017-10-01,02:30,290.289
2017-10-01,03:00,290.289
2017-10-01,03:30,290.289
2017-10-01,04:00,290.28

   These commands read the file and convert the date and elev columns:
wy2018 <- read.table('sh-2018.dat', header = T, sep = ',')
wy2018$date <- as.Date(as.character(wy2018$date, format='y-m-d'))
head(wy2018)
         date time    elev
1 2017-10-01   01 290.298
2 2017-10-01   01 290.301
3 2017-10-01   01 290.304
4 2017-10-01   01 290.295
5 2017-10-01   01 290.292
6 2017-10-01   01 290.289

   My attempts using chron() for the time column keep failing; e.g.,
wy2018$time <- chron(wy2018$time, format='h:m')
str(wy2018)
'data.frame': 12592 obs. of  3 variables:
  $ date: Date, format: "2017-10-01" "2017-10-01" ...
  $ time: 'dates' num  01 01 01 01 01 01 01 01 01 01 ...
   ..- attr(*, "format")= chr "h:m"
   ..- attr(*, "origin")= Named num  1 1 1970
   .. ..- attr(*, "names")= chr  "month" "day" "year"
  $ elev: num  290 290 290 290 290 ...

   Also, when I tried to use chron() for both the date and time columns of
the dataframe these failed, too.

   Please teach me how to read the data sources and produce dataframe columns
of date, time, and numeric.

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

Rich Shepard
On Thu, 19 Jul 2018, Rich Shepard wrote:

> I have the date and elev columns converted from factors to date and
> numeric, respectively, but still have not learned how to convert the time.

   With this dataframe structure,

str(wy2018)
'data.frame': 12592 obs. of  3 variables:
  $ date: Date, format: "2017-10-01" "2017-10-01" ...
  $ time: chr  "00:00" "00:30" "01:00" "01:30" ...
  $ elev: num  290 290 290 290 290 ...

what is my syntax error using chron() to convert the time column?

wy2018$time <- chron(times=wy2018$time)
Error in convert.times(times., fmt) : format h:m:s may be incorrect
In addition: Warning message:
In unpaste(times, sep = fmt$sep, fnames = fmt$periods, nfields = 3) :
   12592 entries set to NA due to wrong number of fields


   Adding a format as either %h:%m or just h:m makes no difference.

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

David Winsemius
In reply to this post by Rich Shepard

> On Jul 19, 2018, at 10:02 AM, Rich Shepard <[hidden email]> wrote:
>
> On Thu, 19 Jul 2018, Rich Shepard wrote:
>
>> Since then I reformatted the file to two fields: date-time and elevation.
>> If anyone wants a copy send me a message off the list and I'll respond with
>> the modified file attached.
>
>  This is a mistake. The file needs commas separating each field.
>
>  I have the date and elev columns converted from factors to date and
> numeric, respectively, but still have not learned how to convert the time.
>
>  The source data file (head):
> date,time,elev
> 2017-10-01,00:00,290.298
> 2017-10-01,00:30,290.301
> 2017-10-01,01:00,290.304
> 2017-10-01,01:30,290.295
> 2017-10-01,02:00,290.292
> 2017-10-01,02:30,290.289
> 2017-10-01,03:00,290.289
> 2017-10-01,03:30,290.289
> 2017-10-01,04:00,290.28

I took the code that I offered earlier and replaced allyears with wy2018:

> txt <- "date,time,elev
+ 2017-10-01,00:00,290.298
+ 2017-10-01,00:30,290.301
+ 2017-10-01,01:00,290.304
+ 2017-10-01,01:30,290.295
+ 2017-10-01,02:00,290.292
+ 2017-10-01,02:30,290.289
+ 2017-10-01,03:00,290.289
+ 2017-10-01,03:30,290.289

+ 2017-10-01,04:00,290.28"
> wy2018 <- read.table(text=txt, header = T, sep = ',')

> wy2018 $myDate <- as.Date(as.character(wy2018 $date))
> wy2018 $myTime <- as.POSIXct(paste(wy2018 $date, wy2018 $time))
> wy2018
        date  time    elev     myDate              myTime
1 2017-10-01 00:00 290.298 2017-10-01 2017-10-01 00:00:00
2 2017-10-01 00:30 290.301 2017-10-01 2017-10-01 00:30:00
3 2017-10-01 01:00 290.304 2017-10-01 2017-10-01 01:00:00
4 2017-10-01 01:30 290.295 2017-10-01 2017-10-01 01:30:00
5 2017-10-01 02:00 290.292 2017-10-01 2017-10-01 02:00:00
6 2017-10-01 02:30 290.289 2017-10-01 2017-10-01 02:30:00
7 2017-10-01 03:00 290.289 2017-10-01 2017-10-01 03:00:00
8 2017-10-01 03:30 290.289 2017-10-01 2017-10-01 03:30:00
9 2017-10-01 04:00 290.280 2017-10-01 2017-10-01 04:00:00

> str(wy2018)
'data.frame': 9 obs. of  5 variables:
 $ date  : Factor w/ 1 level "2017-10-01": 1 1 1 1 1 1 1 1 1
 $ time  : Factor w/ 9 levels "00:00","00:30",..: 1 2 3 4 5 6 7 8 9
 $ elev  : num  290 290 290 290 290 ...
 $ myDate: Date, format: "2017-10-01" "2017-10-01" ...
 $ myTime: POSIXct, format: "2017-10-01 00:00:00" "2017-10-01 00:30:00" ...

>
>  These commands read the file and convert the date and elev columns:
> wy2018 <- read.table('sh-2018.dat', header = T, sep = ',')
> wy2018$date <- as.Date(as.character(wy2018$date, format='y-m-d'))
> head(wy2018)
>        date time    elev
> 1 2017-10-01   01 290.298
> 2 2017-10-01   01 290.301
> 3 2017-10-01   01 290.304
> 4 2017-10-01   01 290.295
> 5 2017-10-01   01 290.292
> 6 2017-10-01   01 290.289
>
>  My attempts using chron() for the time column keep failing; e.g., wy2018$time <- chron(wy2018$time, format='h:m')
> str(wy2018)
> 'data.frame': 12592 obs. of  3 variables:
> $ date: Date, format: "2017-10-01" "2017-10-01" ...
> $ time: 'dates' num  01 01 01 01 01 01 01 01 01 01 ...
>  ..- attr(*, "format")= chr "h:m"
>  ..- attr(*, "origin")= Named num  1 1 1970
>  .. ..- attr(*, "names")= chr  "month" "day" "year"
> $ elev: num  290 290 290 290 290 ...
>
>  Also, when I tried to use chron() for both the date and time columns of
> the dataframe these failed, too.
>
>  Please teach me how to read the data sources and produce dataframe columns
> of date, time, and numeric.
>
> Rich
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.'   -Gehm's Corollary to Clarke's Third Law

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors

David Winsemius
In reply to this post by Rich Shepard

> On Jul 19, 2018, at 12:33 PM, Rich Shepard <[hidden email]> wrote:
>
> On Thu, 19 Jul 2018, Rich Shepard wrote:
>
>> I have the date and elev columns converted from factors to date and
>> numeric, respectively, but still have not learned how to convert the time.
>
>  With this dataframe structure,
>
> str(wy2018)
> 'data.frame': 12592 obs. of  3 variables:
> $ date: Date, format: "2017-10-01" "2017-10-01" ...
> $ time: chr  "00:00" "00:30" "01:00" "01:30" ...
> $ elev: num  290 290 290 290 290 ...
>
> what is my syntax error using chron() to convert the time column?

I have no idea. I think reaching for chron is not needed.
>
> wy2018$time <- chron(times=wy2018$time)
> Error in convert.times(times., fmt) : format h:m:s may be incorrect
> In addition: Warning message:
> In unpaste(times, sep = fmt$sep, fnames = fmt$periods, nfields = 3) :
>  12592 entries set to NA due to wrong number of fields

That error suggests you needed either to add ":00" on hte end of your hh:mm formated data or provide a format string.

> wy2018$myTime <- chron(times= paste(wy2018$time, ":00") )


> str(wy2018)
'data.frame': 9 obs. of  5 variables:
 $ date  : Factor w/ 1 level "2017-10-01": 1 1 1 1 1 1 1 1 1
 $ time  : Factor w/ 9 levels "00:00","00:30",..: 1 2 3 4 5 6 7 8 9
 $ elev  : num  290 290 290 290 290 ...
 $ myDate: Date, format: "2017-10-01" "2017-10-01" ...
 $ myTime: 'times' num  00:00:00 00:30:00 01:00:00 01:30:00 02:00:00 ...
  ..- attr(*, "format")= chr "h:m:s"

But I think that is not the right way to go.

>
>
>  Adding a format as either %h:%m or just h:m makes no difference.
>
> Rich
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.'   -Gehm's Corollary to Clarke's Third Law

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors [RESOLVED]

Rich Shepard
In reply to this post by David Winsemius
On Thu, 19 Jul 2018, David Winsemius wrote:

> I took the code that I offered earlier and replaced allyears with wy2018:

>        date  time    elev     myDate              myTime
> 1 2017-10-01 00:00 290.298 2017-10-01 2017-10-01 00:00:00
> 2 2017-10-01 00:30 290.301 2017-10-01 2017-10-01 00:30:00
> 3 2017-10-01 01:00 290.304 2017-10-01 2017-10-01 01:00:00
> 4 2017-10-01 01:30 290.295 2017-10-01 2017-10-01 01:30:00
> 5 2017-10-01 02:00 290.292 2017-10-01 2017-10-01 02:00:00
> 6 2017-10-01 02:30 290.289 2017-10-01 2017-10-01 02:30:00
> 7 2017-10-01 03:00 290.289 2017-10-01 2017-10-01 03:00:00
> 8 2017-10-01 03:30 290.289 2017-10-01 2017-10-01 03:30:00
> 9 2017-10-01 04:00 290.280 2017-10-01 2017-10-01 04:00:00
>
>> str(wy2018)
> 'data.frame': 9 obs. of  5 variables:
> $ date  : Factor w/ 1 level "2017-10-01": 1 1 1 1 1 1 1 1 1
> $ time  : Factor w/ 9 levels "00:00","00:30",..: 1 2 3 4 5 6 7 8 9
> $ elev  : num  290 290 290 290 290 ...
> $ myDate: Date, format: "2017-10-01" "2017-10-01" ...
> $ myTime: POSIXct, format: "2017-10-01 00:00:00" "2017-10-01 00:30:00" ...

David,

   Thank you. I see the results in the dataframe structure although I still
don't understand all the reasons. The 'myTime' column confirms what I
thought: that there is no separate time data type. I'll use what you taught
me an move on with the analyses.

Best regards,

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors [RESOLVED]

David Winsemius

> On Jul 19, 2018, at 1:21 PM, Rich Shepard <[hidden email]> wrote:
>
> On Thu, 19 Jul 2018, David Winsemius wrote:
>
>> I took the code that I offered earlier and replaced allyears with wy2018:
>
>>       date  time    elev     myDate              myTime
>> 1 2017-10-01 00:00 290.298 2017-10-01 2017-10-01 00:00:00
>> 2 2017-10-01 00:30 290.301 2017-10-01 2017-10-01 00:30:00
>> 3 2017-10-01 01:00 290.304 2017-10-01 2017-10-01 01:00:00
>> 4 2017-10-01 01:30 290.295 2017-10-01 2017-10-01 01:30:00
>> 5 2017-10-01 02:00 290.292 2017-10-01 2017-10-01 02:00:00
>> 6 2017-10-01 02:30 290.289 2017-10-01 2017-10-01 02:30:00
>> 7 2017-10-01 03:00 290.289 2017-10-01 2017-10-01 03:00:00
>> 8 2017-10-01 03:30 290.289 2017-10-01 2017-10-01 03:30:00
>> 9 2017-10-01 04:00 290.280 2017-10-01 2017-10-01 04:00:00
>>
>>> str(wy2018)
>> 'data.frame': 9 obs. of  5 variables:
>> $ date  : Factor w/ 1 level "2017-10-01": 1 1 1 1 1 1 1 1 1
>> $ time  : Factor w/ 9 levels "00:00","00:30",..: 1 2 3 4 5 6 7 8 9
>> $ elev  : num  290 290 290 290 290 ...
>> $ myDate: Date, format: "2017-10-01" "2017-10-01" ...
>> $ myTime: POSIXct, format: "2017-10-01 00:00:00" "2017-10-01 00:30:00" ...
>
> David,
>
>  Thank you. I see the results in the dataframe structure although I still
> don't understand all the reasons. The 'myTime' column confirms what I
> thought: that there is no separate time data type. I'll use what you taught
> me an move on with the analyses.

You can use format to only display the time portion of a datetime object.

format( Sys.time(), "%H:%M")
[1] "13:57"

You can append the current date to a "time-only" character value and as.POSIXct will do that for you:

as.POSIXct("00:00", format="%H:%M")
[1] "2018-07-19 PDT"

> as.POSIXct(c("00:00", "00:01"), format="%H:%M")
[1] "2018-07-19 00:00:00 PDT" "2018-07-19 00:01:00 PDT"

There is a difftime-class in base R


And the lubridate package defines a duration class. It's not a package I use, and I cannot tell off the top of my head what it thinks the difference might be between a "time-span" and a "duration".

Best of luck.

>
> Best regards,
>
> Rich
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.'   -Gehm's Corollary to Clarke's Third Law

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read in data table, change columns from factors [RESOLVED]

Rich Shepard
On Thu, 19 Jul 2018, David Winsemius wrote:

> You can use format to only display the time portion of a datetime object.
> format( Sys.time(), "%H:%M")
> [1] "13:57"
> You can append the current date to a "time-only" character value and as.POSIXct will do that for you:
> as.POSIXct("00:00", format="%H:%M")
> [1] "2018-07-19 PDT"
> as.POSIXct(c("00:00", "00:01"), format="%H:%M")
> [1] "2018-07-19 00:00:00 PDT" "2018-07-19 00:01:00 PDT"

David,

   I read about these in my R books and on web pages. I kept looking for a
time data type analogous to as.Date, as.character, and as.numeric. That's
what sent me in the wrong direction.

> There is a difftime-class in base R And the lubridate package defines a
> duration class. It's not a package I use, and I cannot tell off the top of
> my head what it thinks the difference might be between a "time-span" and a
> "duration".

   Good to know for the future.

   BTW, in a much earlier response you questioned why there were highly
negative elevations. Good question, because those should have been NAs and I
don't know how I let them in the data I manually extracted from PDF reports
from the USGS.

Best regards,

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.