read txt file - date - no space

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

read txt file - date - no space

Diego Avesani
Dear all,

I am dealing with the reading of a *.txt file.
The txt file the following shape:

103001930 103001580 103001530
1998-10-01 00:00:00 0.6 0 0
1998-10-01 01:00:00 0.2 0.2 0.2
1998-10-01 02:00:00 0.6 0.2 0.4
1998-10-01 03:00:00 0 0 0.6
1998-10-01 04:00:00 0 0 0
1998-10-01 05:00:00 0 0 0
1998-10-01 06:00:00 0 0 0
1998-10-01 07:00:00 0.2 0 0

If it is possible I have a coupe of questions, which will sound stupid but
they are important to me in order to understand ho R deal with file or date.

1) Do I have to convert it to a *csv file?
2) Can a deal with space and not ","
3) How can I read date?

thanks a lot to all of you,
Thanks


Diego

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Jeff Newmiller
1) No.

2) The read.csv function is a s special case use of the more general read.table function that can handle any simple field separator.

3) Read the data in as character (I recommend using the stringsAsFactors=FALSE argument to read.table) and convert to an appropriate type from that form. e.g. [1]

[1] https://www.r-bloggers.com/using-dates-and-times-in-r/


On July 30, 2018 7:29:05 AM PDT, Diego Avesani <[hidden email]> wrote:

>Dear all,
>
>I am dealing with the reading of a *.txt file.
>The txt file the following shape:
>
>103001930 103001580 103001530
>1998-10-01 00:00:00 0.6 0 0
>1998-10-01 01:00:00 0.2 0.2 0.2
>1998-10-01 02:00:00 0.6 0.2 0.4
>1998-10-01 03:00:00 0 0 0.6
>1998-10-01 04:00:00 0 0 0
>1998-10-01 05:00:00 0 0 0
>1998-10-01 06:00:00 0 0 0
>1998-10-01 07:00:00 0.2 0 0
>
>If it is possible I have a coupe of questions, which will sound stupid
>but
>they are important to me in order to understand ho R deal with file or
>date.
>
>1) Do I have to convert it to a *csv file?
>2) Can a deal with space and not ","
>3) How can I read date?
>
>thanks a lot to all of you,
>Thanks
>
>
>Diego
>
> [[alternative HTML version deleted]]
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Jim Lemon-4
In reply to this post by Diego Avesani
Hi Diego,
You may have to do some conversion as you have three fields in the
first line using the default space separator and five fields in
subsequent lines. If the first line doesn't contain any important data
you can just delete it or replace it with a meaningful header line
with five fields and save the file under another name.

It looks as thought you have date-time as two fields. If so, you can
just read the first field if you only want the date:

# assume you have removed the first line
dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")

If you want the date/time:

dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d %H:%M:%S")

Jim

On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani <[hidden email]> wrote:

> Dear all,
>
> I am dealing with the reading of a *.txt file.
> The txt file the following shape:
>
> 103001930 103001580 103001530
> 1998-10-01 00:00:00 0.6 0 0
> 1998-10-01 01:00:00 0.2 0.2 0.2
> 1998-10-01 02:00:00 0.6 0.2 0.4
> 1998-10-01 03:00:00 0 0 0.6
> 1998-10-01 04:00:00 0 0 0
> 1998-10-01 05:00:00 0 0 0
> 1998-10-01 06:00:00 0 0 0
> 1998-10-01 07:00:00 0.2 0 0
>
> If it is possible I have a coupe of questions, which will sound stupid but
> they are important to me in order to understand ho R deal with file or date.
>
> 1) Do I have to convert it to a *csv file?
> 2) Can a deal with space and not ","
> 3) How can I read date?
>
> thanks a lot to all of you,
> Thanks
>
>
> Diego
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

R help mailing list-2
Or, without removing the first line
  dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)

Another alternative,
   dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
since the dates appear to be in the default format.
(I generally prefer to work with datetimes in POSIXct class rather than POSIXlt class)

-Don

--
Don MacQueen
Lawrence Livermore National Laboratory
7000 East Ave., L-627
Livermore, CA 94550
925-423-1062
Lab cell 925-724-7509
 
 

On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon" <[hidden email] on behalf of [hidden email]> wrote:

    Hi Diego,
    You may have to do some conversion as you have three fields in the
    first line using the default space separator and five fields in
    subsequent lines. If the first line doesn't contain any important data
    you can just delete it or replace it with a meaningful header line
    with five fields and save the file under another name.
   
    It looks as thought you have date-time as two fields. If so, you can
    just read the first field if you only want the date:
   
    # assume you have removed the first line
    dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
    dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
   
    If you want the date/time:
   
    dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d %H:%M:%S")
   
    Jim
   
    On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani <[hidden email]> wrote:
    > Dear all,
    >
    > I am dealing with the reading of a *.txt file.
    > The txt file the following shape:
    >
    > 103001930 103001580 103001530
    > 1998-10-01 00:00:00 0.6 0 0
    > 1998-10-01 01:00:00 0.2 0.2 0.2
    > 1998-10-01 02:00:00 0.6 0.2 0.4
    > 1998-10-01 03:00:00 0 0 0.6
    > 1998-10-01 04:00:00 0 0 0
    > 1998-10-01 05:00:00 0 0 0
    > 1998-10-01 06:00:00 0 0 0
    > 1998-10-01 07:00:00 0.2 0 0
    >
    > If it is possible I have a coupe of questions, which will sound stupid but
    > they are important to me in order to understand ho R deal with file or date.
    >
    > 1) Do I have to convert it to a *csv file?
    > 2) Can a deal with space and not ","
    > 3) How can I read date?
    >
    > thanks a lot to all of you,
    > Thanks
    >
    >
    > Diego
    >
    >         [[alternative HTML version deleted]]
    >
    > ______________________________________________
    > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
    > https://stat.ethz.ch/mailman/listinfo/r-help
    > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
    > and provide commented, minimal, self-contained, reproducible code.
   
    ______________________________________________
    [hidden email] mailing list -- To UNSUBSCRIBE and more, see
    https://stat.ethz.ch/mailman/listinfo/r-help
    PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
    and provide commented, minimal, self-contained, reproducible code.
   

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear all,
I move to csv file because originally the date where in csv file.
In addition, due to the fact that, as you told me, read.csv is a special
case of read.table, I prefer start to learn from the simplest one.
After that, I will try also the *.txt format.

with read.csv, something strange happened:

This us now the file:

date,st1,st2,st3,
10/1/1998 0:00,0.6,0,0
10/1/1998 1:00,0.2,0.2,0.2
10/1/1998 2:00,0.6,0.2,0.4
10/1/1998 3:00,0,0,0.6
10/1/1998 4:00,0,0,0
10/1/1998 5:00,0,0,0
10/1/1998 6:00,0,0,0
10/1/1998 7:00,0.2,0,0
10/1/1998 8:00,0.6,0.2,0
10/1/1998 9:00,0.2,0.4,0.4
10/1/1998 10:00,0,0.4,0.2

When I apply:
MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")

this is the results:

10/1/1998 0:00    0.6    0.00    0.0 NA
2        10/1/1998 1:00    0.2    0.20    0.2 NA
3        10/1/1998 2:00    0.6    0.20    0.4 NA
4        10/1/1998 3:00    0.0    0.00    0.6 NA
5        10/1/1998 4:00    0.0    0.00    0.0 NA
6        10/1/1998 5:00    0.0    0.00    0.0 NA
7        10/1/1998 6:00    0.0    0.00    0.0 NA
8        10/1/1998 7:00    0.2    0.00    0.0 NA

I do not understand why.
Something wrong with date?

really really thanks,
I appreciate a lot all your helps.

Diedro


Diego


On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]> wrote:

> Or, without removing the first line
>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
>
> Another alternative,
>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> since the dates appear to be in the default format.
> (I generally prefer to work with datetimes in POSIXct class rather than
> POSIXlt class)
>
> -Don
>
> --
> Don MacQueen
> Lawrence Livermore National Laboratory
> 7000 East Ave., L-627
> Livermore, CA 94550
> 925-423-1062
> Lab cell 925-724-7509
>
>
>
> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon" <
> [hidden email] on behalf of [hidden email]> wrote:
>
>     Hi Diego,
>     You may have to do some conversion as you have three fields in the
>     first line using the default space separator and five fields in
>     subsequent lines. If the first line doesn't contain any important data
>     you can just delete it or replace it with a meaningful header line
>     with five fields and save the file under another name.
>
>     It looks as thought you have date-time as two fields. If so, you can
>     just read the first field if you only want the date:
>
>     # assume you have removed the first line
>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>
>     If you want the date/time:
>
>     dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> %H:%M:%S")
>
>     Jim
>
>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani <
> [hidden email]> wrote:
>     > Dear all,
>     >
>     > I am dealing with the reading of a *.txt file.
>     > The txt file the following shape:
>     >
>     > 103001930 103001580 103001530
>     > 1998-10-01 00:00:00 0.6 0 0
>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>     > 1998-10-01 03:00:00 0 0 0.6
>     > 1998-10-01 04:00:00 0 0 0
>     > 1998-10-01 05:00:00 0 0 0
>     > 1998-10-01 06:00:00 0 0 0
>     > 1998-10-01 07:00:00 0.2 0 0
>     >
>     > If it is possible I have a coupe of questions, which will sound
> stupid but
>     > they are important to me in order to understand ho R deal with file
> or date.
>     >
>     > 1) Do I have to convert it to a *csv file?
>     > 2) Can a deal with space and not ","
>     > 3) How can I read date?
>     >
>     > thanks a lot to all of you,
>     > Thanks
>     >
>     >
>     > Diego
>     >
>     >         [[alternative HTML version deleted]]
>     >
>     > ______________________________________________
>     > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>     > https://stat.ethz.ch/mailman/listinfo/r-help
>     > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
>     > and provide commented, minimal, self-contained, reproducible code.
>
>     ______________________________________________
>     [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>     https://stat.ethz.ch/mailman/listinfo/r-help
>     PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
>     and provide commented, minimal, self-contained, reproducible code.
>
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear all,
I have found the error, my fault. Sorry.
There was an extra come in the headers line.
Thanks again.

If I can I would like to ask you another questions about the imported data.
I would like to compute the daily average of the different date. Basically
I have hourly data, I would like to ave the daily mean of them.

Is there some special commands?

Thanks a lot.


Diego


On 31 July 2018 at 10:40, Diego Avesani <[hidden email]> wrote:

> Dear all,
> I move to csv file because originally the date where in csv file.
> In addition, due to the fact that, as you told me, read.csv is a special
> case of read.table, I prefer start to learn from the simplest one.
> After that, I will try also the *.txt format.
>
> with read.csv, something strange happened:
>
> This us now the file:
>
> date,st1,st2,st3,
> 10/1/1998 0:00,0.6,0,0
> 10/1/1998 1:00,0.2,0.2,0.2
> 10/1/1998 2:00,0.6,0.2,0.4
> 10/1/1998 3:00,0,0,0.6
> 10/1/1998 4:00,0,0,0
> 10/1/1998 5:00,0,0,0
> 10/1/1998 6:00,0,0,0
> 10/1/1998 7:00,0.2,0,0
> 10/1/1998 8:00,0.6,0.2,0
> 10/1/1998 9:00,0.2,0.4,0.4
> 10/1/1998 10:00,0,0.4,0.2
>
> When I apply:
> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>
> this is the results:
>
> 10/1/1998 0:00    0.6    0.00    0.0 NA
> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
>
> I do not understand why.
> Something wrong with date?
>
> really really thanks,
> I appreciate a lot all your helps.
>
> Diedro
>
>
> Diego
>
>
> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]> wrote:
>
>> Or, without removing the first line
>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
>>
>> Another alternative,
>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>> since the dates appear to be in the default format.
>> (I generally prefer to work with datetimes in POSIXct class rather than
>> POSIXlt class)
>>
>> -Don
>>
>> --
>> Don MacQueen
>> Lawrence Livermore National Laboratory
>> 7000 East Ave., L-627
>> Livermore, CA 94550
>> 925-423-1062
>> Lab cell 925-724-7509
>>
>>
>>
>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon" <
>> [hidden email] on behalf of [hidden email]> wrote:
>>
>>     Hi Diego,
>>     You may have to do some conversion as you have three fields in the
>>     first line using the default space separator and five fields in
>>     subsequent lines. If the first line doesn't contain any important data
>>     you can just delete it or replace it with a meaningful header line
>>     with five fields and save the file under another name.
>>
>>     It looks as thought you have date-time as two fields. If so, you can
>>     just read the first field if you only want the date:
>>
>>     # assume you have removed the first line
>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>>
>>     If you want the date/time:
>>
>>     dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>> %H:%M:%S")
>>
>>     Jim
>>
>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani <
>> [hidden email]> wrote:
>>     > Dear all,
>>     >
>>     > I am dealing with the reading of a *.txt file.
>>     > The txt file the following shape:
>>     >
>>     > 103001930 103001580 103001530
>>     > 1998-10-01 00:00:00 0.6 0 0
>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>>     > 1998-10-01 03:00:00 0 0 0.6
>>     > 1998-10-01 04:00:00 0 0 0
>>     > 1998-10-01 05:00:00 0 0 0
>>     > 1998-10-01 06:00:00 0 0 0
>>     > 1998-10-01 07:00:00 0.2 0 0
>>     >
>>     > If it is possible I have a coupe of questions, which will sound
>> stupid but
>>     > they are important to me in order to understand ho R deal with file
>> or date.
>>     >
>>     > 1) Do I have to convert it to a *csv file?
>>     > 2) Can a deal with space and not ","
>>     > 3) How can I read date?
>>     >
>>     > thanks a lot to all of you,
>>     > Thanks
>>     >
>>     >
>>     > Diego
>>     >
>>     >         [[alternative HTML version deleted]]
>>     >
>>     > ______________________________________________
>>     > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>     > https://stat.ethz.ch/mailman/listinfo/r-help
>>     > PLEASE do read the posting guide http://www.R-project.org/posti
>> ng-guide.html
>>     > and provide commented, minimal, self-contained, reproducible code.
>>
>>     ______________________________________________
>>     [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>     https://stat.ethz.ch/mailman/listinfo/r-help
>>     PLEASE do read the posting guide http://www.R-project.org/posti
>> ng-guide.html
>>     and provide commented, minimal, self-contained, reproducible code.
>>
>>
>>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Jim Lemon-4
Hi Diego,
One way you can get daily means is:

st1_daily<-by(MyData$st1,MyData$date,mean)
st2_daily<-by(MyData$st2,MyData$date,mean)
st3_daily<-by(MyData$st3,MyData$date,mean)

Jim

On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani <[hidden email]> wrote:

> Dear all,
> I have found the error, my fault. Sorry.
> There was an extra come in the headers line.
> Thanks again.
>
> If I can I would like to ask you another questions about the imported data.
> I would like to compute the daily average of the different date. Basically I
> have hourly data, I would like to ave the daily mean of them.
>
> Is there some special commands?
>
> Thanks a lot.
>
>
> Diego
>
>
> On 31 July 2018 at 10:40, Diego Avesani <[hidden email]> wrote:
>>
>> Dear all,
>> I move to csv file because originally the date where in csv file.
>> In addition, due to the fact that, as you told me, read.csv is a special
>> case of read.table, I prefer start to learn from the simplest one.
>> After that, I will try also the *.txt format.
>>
>> with read.csv, something strange happened:
>>
>> This us now the file:
>>
>> date,st1,st2,st3,
>> 10/1/1998 0:00,0.6,0,0
>> 10/1/1998 1:00,0.2,0.2,0.2
>> 10/1/1998 2:00,0.6,0.2,0.4
>> 10/1/1998 3:00,0,0,0.6
>> 10/1/1998 4:00,0,0,0
>> 10/1/1998 5:00,0,0,0
>> 10/1/1998 6:00,0,0,0
>> 10/1/1998 7:00,0.2,0,0
>> 10/1/1998 8:00,0.6,0.2,0
>> 10/1/1998 9:00,0.2,0.4,0.4
>> 10/1/1998 10:00,0,0.4,0.2
>>
>> When I apply:
>> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>>
>> this is the results:
>>
>> 10/1/1998 0:00    0.6    0.00    0.0 NA
>> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
>> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
>> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
>> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
>> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
>> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
>> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
>>
>> I do not understand why.
>> Something wrong with date?
>>
>> really really thanks,
>> I appreciate a lot all your helps.
>>
>> Diedro
>>
>>
>> Diego
>>
>>
>> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]> wrote:
>>>
>>> Or, without removing the first line
>>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
>>>
>>> Another alternative,
>>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>>> since the dates appear to be in the default format.
>>> (I generally prefer to work with datetimes in POSIXct class rather than
>>> POSIXlt class)
>>>
>>> -Don
>>>
>>> --
>>> Don MacQueen
>>> Lawrence Livermore National Laboratory
>>> 7000 East Ave., L-627
>>> Livermore, CA 94550
>>> 925-423-1062
>>> Lab cell 925-724-7509
>>>
>>>
>>>
>>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
>>> <[hidden email] on behalf of [hidden email]> wrote:
>>>
>>>     Hi Diego,
>>>     You may have to do some conversion as you have three fields in the
>>>     first line using the default space separator and five fields in
>>>     subsequent lines. If the first line doesn't contain any important
>>> data
>>>     you can just delete it or replace it with a meaningful header line
>>>     with five fields and save the file under another name.
>>>
>>>     It looks as thought you have date-time as two fields. If so, you can
>>>     just read the first field if you only want the date:
>>>
>>>     # assume you have removed the first line
>>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>>>
>>>     If you want the date/time:
>>>
>>>     dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>>> %H:%M:%S")
>>>
>>>     Jim
>>>
>>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
>>> <[hidden email]> wrote:
>>>     > Dear all,
>>>     >
>>>     > I am dealing with the reading of a *.txt file.
>>>     > The txt file the following shape:
>>>     >
>>>     > 103001930 103001580 103001530
>>>     > 1998-10-01 00:00:00 0.6 0 0
>>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>>>     > 1998-10-01 03:00:00 0 0 0.6
>>>     > 1998-10-01 04:00:00 0 0 0
>>>     > 1998-10-01 05:00:00 0 0 0
>>>     > 1998-10-01 06:00:00 0 0 0
>>>     > 1998-10-01 07:00:00 0.2 0 0
>>>     >
>>>     > If it is possible I have a coupe of questions, which will sound
>>> stupid but
>>>     > they are important to me in order to understand ho R deal with file
>>> or date.
>>>     >
>>>     > 1) Do I have to convert it to a *csv file?
>>>     > 2) Can a deal with space and not ","
>>>     > 3) How can I read date?
>>>     >
>>>     > thanks a lot to all of you,
>>>     > Thanks
>>>     >
>>>     >
>>>     > Diego
>>>     >
>>>     >         [[alternative HTML version deleted]]
>>>     >
>>>     > ______________________________________________
>>>     > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>>     > https://stat.ethz.ch/mailman/listinfo/r-help
>>>     > PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>>     > and provide commented, minimal, self-contained, reproducible code.
>>>
>>>     ______________________________________________
>>>     [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>>     https://stat.ethz.ch/mailman/listinfo/r-help
>>>     PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>>     and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>>
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear Jim, Dear all,

thanks a lot.

Unfortunately, I get the following error:


 st1_daily<-by(MyData$st1,MyData$date,mean)Error in
tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L, 925L,  :
  arguments must have same length


This is particularly strange. indeed, if I apply


mean(MyData$str1,na.rm=TRUE)


it works


Sorry, I have to learn a lot.
You are really boosting me

Diego


On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:

> Hi Diego,
> One way you can get daily means is:
>
> st1_daily<-by(MyData$st1,MyData$date,mean)
> st2_daily<-by(MyData$st2,MyData$date,mean)
> st3_daily<-by(MyData$st3,MyData$date,mean)
>
> Jim
>
> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani <[hidden email]>
> wrote:
> > Dear all,
> > I have found the error, my fault. Sorry.
> > There was an extra come in the headers line.
> > Thanks again.
> >
> > If I can I would like to ask you another questions about the imported
> data.
> > I would like to compute the daily average of the different date.
> Basically I
> > have hourly data, I would like to ave the daily mean of them.
> >
> > Is there some special commands?
> >
> > Thanks a lot.
> >
> >
> > Diego
> >
> >
> > On 31 July 2018 at 10:40, Diego Avesani <[hidden email]> wrote:
> >>
> >> Dear all,
> >> I move to csv file because originally the date where in csv file.
> >> In addition, due to the fact that, as you told me, read.csv is a special
> >> case of read.table, I prefer start to learn from the simplest one.
> >> After that, I will try also the *.txt format.
> >>
> >> with read.csv, something strange happened:
> >>
> >> This us now the file:
> >>
> >> date,st1,st2,st3,
> >> 10/1/1998 0:00,0.6,0,0
> >> 10/1/1998 1:00,0.2,0.2,0.2
> >> 10/1/1998 2:00,0.6,0.2,0.4
> >> 10/1/1998 3:00,0,0,0.6
> >> 10/1/1998 4:00,0,0,0
> >> 10/1/1998 5:00,0,0,0
> >> 10/1/1998 6:00,0,0,0
> >> 10/1/1998 7:00,0.2,0,0
> >> 10/1/1998 8:00,0.6,0.2,0
> >> 10/1/1998 9:00,0.2,0.4,0.4
> >> 10/1/1998 10:00,0,0.4,0.2
> >>
> >> When I apply:
> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> >>
> >> this is the results:
> >>
> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
> >>
> >> I do not understand why.
> >> Something wrong with date?
> >>
> >> really really thanks,
> >> I appreciate a lot all your helps.
> >>
> >> Diedro
> >>
> >>
> >> Diego
> >>
> >>
> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]> wrote:
> >>>
> >>> Or, without removing the first line
> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
> >>>
> >>> Another alternative,
> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> >>> since the dates appear to be in the default format.
> >>> (I generally prefer to work with datetimes in POSIXct class rather than
> >>> POSIXlt class)
> >>>
> >>> -Don
> >>>
> >>> --
> >>> Don MacQueen
> >>> Lawrence Livermore National Laboratory
> >>> 7000 East Ave., L-627
> >>> Livermore, CA 94550
> >>> 925-423-1062
> >>> Lab cell 925-724-7509
> >>>
> >>>
> >>>
> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
> >>> <[hidden email] on behalf of [hidden email]>
> wrote:
> >>>
> >>>     Hi Diego,
> >>>     You may have to do some conversion as you have three fields in the
> >>>     first line using the default space separator and five fields in
> >>>     subsequent lines. If the first line doesn't contain any important
> >>> data
> >>>     you can just delete it or replace it with a meaningful header line
> >>>     with five fields and save the file under another name.
> >>>
> >>>     It looks as thought you have date-time as two fields. If so, you
> can
> >>>     just read the first field if you only want the date:
> >>>
> >>>     # assume you have removed the first line
> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
> >>>
> >>>     If you want the date/time:
> >>>
> >>>     dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> >>> %H:%M:%S")
> >>>
> >>>     Jim
> >>>
> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
> >>> <[hidden email]> wrote:
> >>>     > Dear all,
> >>>     >
> >>>     > I am dealing with the reading of a *.txt file.
> >>>     > The txt file the following shape:
> >>>     >
> >>>     > 103001930 103001580 103001530
> >>>     > 1998-10-01 00:00:00 0.6 0 0
> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
> >>>     > 1998-10-01 03:00:00 0 0 0.6
> >>>     > 1998-10-01 04:00:00 0 0 0
> >>>     > 1998-10-01 05:00:00 0 0 0
> >>>     > 1998-10-01 06:00:00 0 0 0
> >>>     > 1998-10-01 07:00:00 0.2 0 0
> >>>     >
> >>>     > If it is possible I have a coupe of questions, which will sound
> >>> stupid but
> >>>     > they are important to me in order to understand ho R deal with
> file
> >>> or date.
> >>>     >
> >>>     > 1) Do I have to convert it to a *csv file?
> >>>     > 2) Can a deal with space and not ","
> >>>     > 3) How can I read date?
> >>>     >
> >>>     > thanks a lot to all of you,
> >>>     > Thanks
> >>>     >
> >>>     >
> >>>     > Diego
> >>>     >
> >>>     >         [[alternative HTML version deleted]]
> >>>     >
> >>>     > ______________________________________________
> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and more,
> see
> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
> >>>     > PLEASE do read the posting guide
> >>> http://www.R-project.org/posting-guide.html
> >>>     > and provide commented, minimal, self-contained, reproducible
> code.
> >>>
> >>>     ______________________________________________
> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
> >>>     PLEASE do read the posting guide
> >>> http://www.R-project.org/posting-guide.html
> >>>     and provide commented, minimal, self-contained, reproducible code.
> >>>
> >>>
> >>
> >
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear all,

I have still problem with date.
Could you please tel me how to use POSIXct.
Indeed I have found this command:
timeAverage, but I am not able to convert MyDate to properly date.

Thank a lot
I hope to no bother you, at least too much


Diego


On 31 July 2018 at 11:12, Diego Avesani <[hidden email]> wrote:

> Dear Jim, Dear all,
>
> thanks a lot.
>
> Unfortunately, I get the following error:
>
>
>  st1_daily<-by(MyData$st1,MyData$date,mean)Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L, 925L,  :
>   arguments must have same length
>
>
> This is particularly strange. indeed, if I apply
>
>
> mean(MyData$str1,na.rm=TRUE)
>
>
> it works
>
>
> Sorry, I have to learn a lot.
> You are really boosting me
>
> Diego
>
>
> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
>
>> Hi Diego,
>> One way you can get daily means is:
>>
>> st1_daily<-by(MyData$st1,MyData$date,mean)
>> st2_daily<-by(MyData$st2,MyData$date,mean)
>> st3_daily<-by(MyData$st3,MyData$date,mean)
>>
>> Jim
>>
>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani <[hidden email]>
>> wrote:
>> > Dear all,
>> > I have found the error, my fault. Sorry.
>> > There was an extra come in the headers line.
>> > Thanks again.
>> >
>> > If I can I would like to ask you another questions about the imported
>> data.
>> > I would like to compute the daily average of the different date.
>> Basically I
>> > have hourly data, I would like to ave the daily mean of them.
>> >
>> > Is there some special commands?
>> >
>> > Thanks a lot.
>> >
>> >
>> > Diego
>> >
>> >
>> > On 31 July 2018 at 10:40, Diego Avesani <[hidden email]>
>> wrote:
>> >>
>> >> Dear all,
>> >> I move to csv file because originally the date where in csv file.
>> >> In addition, due to the fact that, as you told me, read.csv is a
>> special
>> >> case of read.table, I prefer start to learn from the simplest one.
>> >> After that, I will try also the *.txt format.
>> >>
>> >> with read.csv, something strange happened:
>> >>
>> >> This us now the file:
>> >>
>> >> date,st1,st2,st3,
>> >> 10/1/1998 0:00,0.6,0,0
>> >> 10/1/1998 1:00,0.2,0.2,0.2
>> >> 10/1/1998 2:00,0.6,0.2,0.4
>> >> 10/1/1998 3:00,0,0,0.6
>> >> 10/1/1998 4:00,0,0,0
>> >> 10/1/1998 5:00,0,0,0
>> >> 10/1/1998 6:00,0,0,0
>> >> 10/1/1998 7:00,0.2,0,0
>> >> 10/1/1998 8:00,0.6,0.2,0
>> >> 10/1/1998 9:00,0.2,0.4,0.4
>> >> 10/1/1998 10:00,0,0.4,0.2
>> >>
>> >> When I apply:
>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>> >>
>> >> this is the results:
>> >>
>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
>> >>
>> >> I do not understand why.
>> >> Something wrong with date?
>> >>
>> >> really really thanks,
>> >> I appreciate a lot all your helps.
>> >>
>> >> Diedro
>> >>
>> >>
>> >> Diego
>> >>
>> >>
>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]> wrote:
>> >>>
>> >>> Or, without removing the first line
>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
>> >>>
>> >>> Another alternative,
>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>> >>> since the dates appear to be in the default format.
>> >>> (I generally prefer to work with datetimes in POSIXct class rather
>> than
>> >>> POSIXlt class)
>> >>>
>> >>> -Don
>> >>>
>> >>> --
>> >>> Don MacQueen
>> >>> Lawrence Livermore National Laboratory
>> >>> 7000 East Ave., L-627
>> >>> Livermore, CA 94550
>> >>> 925-423-1062
>> >>> Lab cell 925-724-7509
>> >>>
>> >>>
>> >>>
>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
>> >>> <[hidden email] on behalf of [hidden email]>
>> wrote:
>> >>>
>> >>>     Hi Diego,
>> >>>     You may have to do some conversion as you have three fields in the
>> >>>     first line using the default space separator and five fields in
>> >>>     subsequent lines. If the first line doesn't contain any important
>> >>> data
>> >>>     you can just delete it or replace it with a meaningful header line
>> >>>     with five fields and save the file under another name.
>> >>>
>> >>>     It looks as thought you have date-time as two fields. If so, you
>> can
>> >>>     just read the first field if you only want the date:
>> >>>
>> >>>     # assume you have removed the first line
>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>> >>>
>> >>>     If you want the date/time:
>> >>>
>> >>>     dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>> >>> %H:%M:%S")
>> >>>
>> >>>     Jim
>> >>>
>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
>> >>> <[hidden email]> wrote:
>> >>>     > Dear all,
>> >>>     >
>> >>>     > I am dealing with the reading of a *.txt file.
>> >>>     > The txt file the following shape:
>> >>>     >
>> >>>     > 103001930 103001580 103001530
>> >>>     > 1998-10-01 00:00:00 0.6 0 0
>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>> >>>     > 1998-10-01 03:00:00 0 0 0.6
>> >>>     > 1998-10-01 04:00:00 0 0 0
>> >>>     > 1998-10-01 05:00:00 0 0 0
>> >>>     > 1998-10-01 06:00:00 0 0 0
>> >>>     > 1998-10-01 07:00:00 0.2 0 0
>> >>>     >
>> >>>     > If it is possible I have a coupe of questions, which will sound
>> >>> stupid but
>> >>>     > they are important to me in order to understand ho R deal with
>> file
>> >>> or date.
>> >>>     >
>> >>>     > 1) Do I have to convert it to a *csv file?
>> >>>     > 2) Can a deal with space and not ","
>> >>>     > 3) How can I read date?
>> >>>     >
>> >>>     > thanks a lot to all of you,
>> >>>     > Thanks
>> >>>     >
>> >>>     >
>> >>>     > Diego
>> >>>     >
>> >>>     >         [[alternative HTML version deleted]]
>> >>>     >
>> >>>     > ______________________________________________
>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and more,
>> see
>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
>> >>>     > PLEASE do read the posting guide
>> >>> http://www.R-project.org/posting-guide.html
>> >>>     > and provide commented, minimal, self-contained, reproducible
>> code.
>> >>>
>> >>>     ______________________________________________
>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
>> >>>     PLEASE do read the posting guide
>> >>> http://www.R-project.org/posting-guide.html
>> >>>     and provide commented, minimal, self-contained, reproducible code.
>> >>>
>> >>>
>> >>
>> >
>>
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Jim Lemon-4
Hi Diego,
I think the error is due to NA values in your data file. If I extend
your example and run it, I get no errors:

MyData<-read.table(text="103001930 103001580 103001530
1998-10-01 00:00:00 0.6 0 0
1998-10-01 01:00:00 0.2 0.2 0.2
1998-10-01 02:00:00 0.6 0.2 0.4
1998-10-01 03:00:00 0 0 0.6
1998-10-01 04:00:00 0 0 0
1998-10-01 05:00:00 0 0 0
1998-10-01 06:00:00 0 0 0
1998-10-01 07:00:00 0.2 0 0
1998-10-01 08:00:00 0.6 0 0
1998-10-01 09:00:00 0.2 0.2 0.2
1998-10-01 10:00:00 0.6 0.2 0.4
1998-10-01 11:00:00 0 0 0.6
1998-10-01 12:00:00 0 0 0
1998-10-01 13:00:00 0 0 0
1998-10-01 14:00:00 0 0 0
1998-10-01 15:00:00 0.2 0 0
1998-10-01 16:00:00 0.6 0 0
1998-10-01 17:00:00 0.2 0.2 0.2
1998-10-01 18:00:00 0.6 0.2 0.4
1998-10-01 19:00:00 0 0 0.6
1998-10-01 20:00:00 0 0 0
1998-10-01 21:00:00 0 0 0
1998-10-01 22:00:00 0 0 0
1998-10-01 23:00:00 0.2 0 0
1998-10-02 00:00:00 0.6 0 0
1998-10-02 01:00:00 0.2 0.2 0.2
1998-10-02 02:00:00 0.6 0.2 0.4
1998-10-02 03:00:00 0 0 0.6
1998-10-02 04:00:00 0 0 0
1998-10-02 05:00:00 0 0 0
1998-10-02 06:00:00 0 0 0
1998-10-02 07:00:00 0.2 0 0
1998-10-02 08:00:00 0.6 0 0
1998-10-02 09:00:00 0.2 0.2 0.2
1998-10-02 10:00:00 0.6 0.2 0.4
1998-10-02 11:00:00 0 0 0.6
1998-10-02 12:00:00 0 0 0
1998-10-02 13:00:00 0 0 0
1998-10-02 14:00:00 0 0 0
1998-10-02 15:00:00 0.2 0 0
1998-10-02 16:00:00 0.6 0 0
1998-10-02 17:00:00 0.2 0.2 0.2
1998-10-02 18:00:00 0.6 0.2 0.4
1998-10-02 19:00:00 0 0 0.6
1998-10-02 20:00:00 0 0 0
1998-10-02 21:00:00 0 0 0
1998-10-02 22:00:00 0 0 0
1998-10-02 23:00:00 0.2 0 0",
skip=1,stringsAsFactors=FALSE)
names(MyData)<-c("date","time","st1","st2","st3")
MyData$datetime<-strptime(paste(MyData$date,MyData$time),
 format="%Y-%m-%d %H:%M:%S")
MyData$datetime
st1_daily<-by(MyData$st1,MyData$date,mean)
st2_daily<-by(MyData$st2,MyData$date,mean)
st3_daily<-by(MyData$st3,MyData$date,mean)
st1_daily
st2_daily
st3_daily

Try adding na.rm=TRUE to the "by" calls:

st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)

Jim

On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani <[hidden email]> wrote:

> Dear all,
>
> I have still problem with date.
> Could you please tel me how to use POSIXct.
> Indeed I have found this command:
> timeAverage, but I am not able to convert MyDate to properly date.
>
> Thank a lot
> I hope to no bother you, at least too much
>
>
> Diego
>
>
> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]> wrote:
>>
>> Dear Jim, Dear all,
>>
>> thanks a lot.
>>
>> Unfortunately, I get the following error:
>>
>>
>>  st1_daily<-by(MyData$st1,MyData$date,mean)
>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L, 925L,  :
>>   arguments must have same length
>>
>>
>> This is particularly strange. indeed, if I apply
>>
>>
>> mean(MyData$str1,na.rm=TRUE)
>>
>>
>> it works
>>
>>
>> Sorry, I have to learn a lot.
>> You are really boosting me
>>
>> Diego
>>
>>
>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
>>>
>>> Hi Diego,
>>> One way you can get daily means is:
>>>
>>> st1_daily<-by(MyData$st1,MyData$date,mean)
>>> st2_daily<-by(MyData$st2,MyData$date,mean)
>>> st3_daily<-by(MyData$st3,MyData$date,mean)
>>>
>>> Jim
>>>
>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani <[hidden email]>
>>> wrote:
>>> > Dear all,
>>> > I have found the error, my fault. Sorry.
>>> > There was an extra come in the headers line.
>>> > Thanks again.
>>> >
>>> > If I can I would like to ask you another questions about the imported
>>> > data.
>>> > I would like to compute the daily average of the different date.
>>> > Basically I
>>> > have hourly data, I would like to ave the daily mean of them.
>>> >
>>> > Is there some special commands?
>>> >
>>> > Thanks a lot.
>>> >
>>> >
>>> > Diego
>>> >
>>> >
>>> > On 31 July 2018 at 10:40, Diego Avesani <[hidden email]>
>>> > wrote:
>>> >>
>>> >> Dear all,
>>> >> I move to csv file because originally the date where in csv file.
>>> >> In addition, due to the fact that, as you told me, read.csv is a
>>> >> special
>>> >> case of read.table, I prefer start to learn from the simplest one.
>>> >> After that, I will try also the *.txt format.
>>> >>
>>> >> with read.csv, something strange happened:
>>> >>
>>> >> This us now the file:
>>> >>
>>> >> date,st1,st2,st3,
>>> >> 10/1/1998 0:00,0.6,0,0
>>> >> 10/1/1998 1:00,0.2,0.2,0.2
>>> >> 10/1/1998 2:00,0.6,0.2,0.4
>>> >> 10/1/1998 3:00,0,0,0.6
>>> >> 10/1/1998 4:00,0,0,0
>>> >> 10/1/1998 5:00,0,0,0
>>> >> 10/1/1998 6:00,0,0,0
>>> >> 10/1/1998 7:00,0.2,0,0
>>> >> 10/1/1998 8:00,0.6,0.2,0
>>> >> 10/1/1998 9:00,0.2,0.4,0.4
>>> >> 10/1/1998 10:00,0,0.4,0.2
>>> >>
>>> >> When I apply:
>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>>> >>
>>> >> this is the results:
>>> >>
>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
>>> >>
>>> >> I do not understand why.
>>> >> Something wrong with date?
>>> >>
>>> >> really really thanks,
>>> >> I appreciate a lot all your helps.
>>> >>
>>> >> Diedro
>>> >>
>>> >>
>>> >> Diego
>>> >>
>>> >>
>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]> wrote:
>>> >>>
>>> >>> Or, without removing the first line
>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
>>> >>>
>>> >>> Another alternative,
>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>>> >>> since the dates appear to be in the default format.
>>> >>> (I generally prefer to work with datetimes in POSIXct class rather
>>> >>> than
>>> >>> POSIXlt class)
>>> >>>
>>> >>> -Don
>>> >>>
>>> >>> --
>>> >>> Don MacQueen
>>> >>> Lawrence Livermore National Laboratory
>>> >>> 7000 East Ave., L-627
>>> >>> Livermore, CA 94550
>>> >>> 925-423-1062
>>> >>> Lab cell 925-724-7509
>>> >>>
>>> >>>
>>> >>>
>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
>>> >>> <[hidden email] on behalf of [hidden email]>
>>> >>> wrote:
>>> >>>
>>> >>>     Hi Diego,
>>> >>>     You may have to do some conversion as you have three fields in
>>> >>> the
>>> >>>     first line using the default space separator and five fields in
>>> >>>     subsequent lines. If the first line doesn't contain any important
>>> >>> data
>>> >>>     you can just delete it or replace it with a meaningful header
>>> >>> line
>>> >>>     with five fields and save the file under another name.
>>> >>>
>>> >>>     It looks as thought you have date-time as two fields. If so, you
>>> >>> can
>>> >>>     just read the first field if you only want the date:
>>> >>>
>>> >>>     # assume you have removed the first line
>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>>> >>>
>>> >>>     If you want the date/time:
>>> >>>
>>> >>>     dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>>> >>> %H:%M:%S")
>>> >>>
>>> >>>     Jim
>>> >>>
>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
>>> >>> <[hidden email]> wrote:
>>> >>>     > Dear all,
>>> >>>     >
>>> >>>     > I am dealing with the reading of a *.txt file.
>>> >>>     > The txt file the following shape:
>>> >>>     >
>>> >>>     > 103001930 103001580 103001530
>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
>>> >>>     > 1998-10-01 04:00:00 0 0 0
>>> >>>     > 1998-10-01 05:00:00 0 0 0
>>> >>>     > 1998-10-01 06:00:00 0 0 0
>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
>>> >>>     >
>>> >>>     > If it is possible I have a coupe of questions, which will sound
>>> >>> stupid but
>>> >>>     > they are important to me in order to understand ho R deal with
>>> >>> file
>>> >>> or date.
>>> >>>     >
>>> >>>     > 1) Do I have to convert it to a *csv file?
>>> >>>     > 2) Can a deal with space and not ","
>>> >>>     > 3) How can I read date?
>>> >>>     >
>>> >>>     > thanks a lot to all of you,
>>> >>>     > Thanks
>>> >>>     >
>>> >>>     >
>>> >>>     > Diego
>>> >>>     >
>>> >>>     >         [[alternative HTML version deleted]]
>>> >>>     >
>>> >>>     > ______________________________________________
>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and more,
>>> >>> see
>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
>>> >>>     > PLEASE do read the posting guide
>>> >>> http://www.R-project.org/posting-guide.html
>>> >>>     > and provide commented, minimal, self-contained, reproducible
>>> >>> code.
>>> >>>
>>> >>>     ______________________________________________
>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
>>> >>>     PLEASE do read the posting guide
>>> >>> http://www.R-project.org/posting-guide.html
>>> >>>     and provide commented, minimal, self-contained, reproducible
>>> >>> code.
>>> >>>
>>> >>>
>>> >>
>>> >
>>
>>
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Jeff Newmiller
... and the most common source of NA values in time data is wrong timezones. You really need to make sure the timezone that is assumed when the character data are converted to POSIXt agrees with the data. In most cases the easiest way to insure this is to use

Sys.setenv(TZ="US/Pacific")

or whatever timezone from

OlsonNames()

corresponds with your data. Execute this setenv function before the strptime or as.POSIXct() function call.

You can use

MyData[ is.na(MyData$datetime), ]

to see which records are failing to convert time.

[1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1

On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]> wrote:

>Hi Diego,
>I think the error is due to NA values in your data file. If I extend
>your example and run it, I get no errors:
>
>MyData<-read.table(text="103001930 103001580 103001530
>1998-10-01 00:00:00 0.6 0 0
>1998-10-01 01:00:00 0.2 0.2 0.2
>1998-10-01 02:00:00 0.6 0.2 0.4
>1998-10-01 03:00:00 0 0 0.6
>1998-10-01 04:00:00 0 0 0
>1998-10-01 05:00:00 0 0 0
>1998-10-01 06:00:00 0 0 0
>1998-10-01 07:00:00 0.2 0 0
>1998-10-01 08:00:00 0.6 0 0
>1998-10-01 09:00:00 0.2 0.2 0.2
>1998-10-01 10:00:00 0.6 0.2 0.4
>1998-10-01 11:00:00 0 0 0.6
>1998-10-01 12:00:00 0 0 0
>1998-10-01 13:00:00 0 0 0
>1998-10-01 14:00:00 0 0 0
>1998-10-01 15:00:00 0.2 0 0
>1998-10-01 16:00:00 0.6 0 0
>1998-10-01 17:00:00 0.2 0.2 0.2
>1998-10-01 18:00:00 0.6 0.2 0.4
>1998-10-01 19:00:00 0 0 0.6
>1998-10-01 20:00:00 0 0 0
>1998-10-01 21:00:00 0 0 0
>1998-10-01 22:00:00 0 0 0
>1998-10-01 23:00:00 0.2 0 0
>1998-10-02 00:00:00 0.6 0 0
>1998-10-02 01:00:00 0.2 0.2 0.2
>1998-10-02 02:00:00 0.6 0.2 0.4
>1998-10-02 03:00:00 0 0 0.6
>1998-10-02 04:00:00 0 0 0
>1998-10-02 05:00:00 0 0 0
>1998-10-02 06:00:00 0 0 0
>1998-10-02 07:00:00 0.2 0 0
>1998-10-02 08:00:00 0.6 0 0
>1998-10-02 09:00:00 0.2 0.2 0.2
>1998-10-02 10:00:00 0.6 0.2 0.4
>1998-10-02 11:00:00 0 0 0.6
>1998-10-02 12:00:00 0 0 0
>1998-10-02 13:00:00 0 0 0
>1998-10-02 14:00:00 0 0 0
>1998-10-02 15:00:00 0.2 0 0
>1998-10-02 16:00:00 0.6 0 0
>1998-10-02 17:00:00 0.2 0.2 0.2
>1998-10-02 18:00:00 0.6 0.2 0.4
>1998-10-02 19:00:00 0 0 0.6
>1998-10-02 20:00:00 0 0 0
>1998-10-02 21:00:00 0 0 0
>1998-10-02 22:00:00 0 0 0
>1998-10-02 23:00:00 0.2 0 0",
>skip=1,stringsAsFactors=FALSE)
>names(MyData)<-c("date","time","st1","st2","st3")
>MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> format="%Y-%m-%d %H:%M:%S")
>MyData$datetime
>st1_daily<-by(MyData$st1,MyData$date,mean)
>st2_daily<-by(MyData$st2,MyData$date,mean)
>st3_daily<-by(MyData$st3,MyData$date,mean)
>st1_daily
>st2_daily
>st3_daily
>
>Try adding na.rm=TRUE to the "by" calls:
>
>st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
>st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
>st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
>
>Jim
>
>On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
><[hidden email]> wrote:
>> Dear all,
>>
>> I have still problem with date.
>> Could you please tel me how to use POSIXct.
>> Indeed I have found this command:
>> timeAverage, but I am not able to convert MyDate to properly date.
>>
>> Thank a lot
>> I hope to no bother you, at least too much
>>
>>
>> Diego
>>
>>
>> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]>
>wrote:
>>>
>>> Dear Jim, Dear all,
>>>
>>> thanks a lot.
>>>
>>> Unfortunately, I get the following error:
>>>
>>>
>>>  st1_daily<-by(MyData$st1,MyData$date,mean)
>>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
>925L,  :
>>>   arguments must have same length
>>>
>>>
>>> This is particularly strange. indeed, if I apply
>>>
>>>
>>> mean(MyData$str1,na.rm=TRUE)
>>>
>>>
>>> it works
>>>
>>>
>>> Sorry, I have to learn a lot.
>>> You are really boosting me
>>>
>>> Diego
>>>
>>>
>>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
>>>>
>>>> Hi Diego,
>>>> One way you can get daily means is:
>>>>
>>>> st1_daily<-by(MyData$st1,MyData$date,mean)
>>>> st2_daily<-by(MyData$st2,MyData$date,mean)
>>>> st3_daily<-by(MyData$st3,MyData$date,mean)
>>>>
>>>> Jim
>>>>
>>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
><[hidden email]>
>>>> wrote:
>>>> > Dear all,
>>>> > I have found the error, my fault. Sorry.
>>>> > There was an extra come in the headers line.
>>>> > Thanks again.
>>>> >
>>>> > If I can I would like to ask you another questions about the
>imported
>>>> > data.
>>>> > I would like to compute the daily average of the different date.
>>>> > Basically I
>>>> > have hourly data, I would like to ave the daily mean of them.
>>>> >
>>>> > Is there some special commands?
>>>> >
>>>> > Thanks a lot.
>>>> >
>>>> >
>>>> > Diego
>>>> >
>>>> >
>>>> > On 31 July 2018 at 10:40, Diego Avesani <[hidden email]>
>>>> > wrote:
>>>> >>
>>>> >> Dear all,
>>>> >> I move to csv file because originally the date where in csv
>file.
>>>> >> In addition, due to the fact that, as you told me, read.csv is a
>>>> >> special
>>>> >> case of read.table, I prefer start to learn from the simplest
>one.
>>>> >> After that, I will try also the *.txt format.
>>>> >>
>>>> >> with read.csv, something strange happened:
>>>> >>
>>>> >> This us now the file:
>>>> >>
>>>> >> date,st1,st2,st3,
>>>> >> 10/1/1998 0:00,0.6,0,0
>>>> >> 10/1/1998 1:00,0.2,0.2,0.2
>>>> >> 10/1/1998 2:00,0.6,0.2,0.4
>>>> >> 10/1/1998 3:00,0,0,0.6
>>>> >> 10/1/1998 4:00,0,0,0
>>>> >> 10/1/1998 5:00,0,0,0
>>>> >> 10/1/1998 6:00,0,0,0
>>>> >> 10/1/1998 7:00,0.2,0,0
>>>> >> 10/1/1998 8:00,0.6,0.2,0
>>>> >> 10/1/1998 9:00,0.2,0.4,0.4
>>>> >> 10/1/1998 10:00,0,0.4,0.2
>>>> >>
>>>> >> When I apply:
>>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>>>> >>
>>>> >> this is the results:
>>>> >>
>>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
>>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
>>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
>>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
>>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
>>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
>>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
>>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
>>>> >>
>>>> >> I do not understand why.
>>>> >> Something wrong with date?
>>>> >>
>>>> >> really really thanks,
>>>> >> I appreciate a lot all your helps.
>>>> >>
>>>> >> Diedro
>>>> >>
>>>> >>
>>>> >> Diego
>>>> >>
>>>> >>
>>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]>
>wrote:
>>>> >>>
>>>> >>> Or, without removing the first line
>>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
>>>> >>>
>>>> >>> Another alternative,
>>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>>>> >>> since the dates appear to be in the default format.
>>>> >>> (I generally prefer to work with datetimes in POSIXct class
>rather
>>>> >>> than
>>>> >>> POSIXlt class)
>>>> >>>
>>>> >>> -Don
>>>> >>>
>>>> >>> --
>>>> >>> Don MacQueen
>>>> >>> Lawrence Livermore National Laboratory
>>>> >>> 7000 East Ave., L-627
>>>> >>> Livermore, CA 94550
>>>> >>> 925-423-1062
>>>> >>> Lab cell 925-724-7509
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
>>>> >>> <[hidden email] on behalf of
>[hidden email]>
>>>> >>> wrote:
>>>> >>>
>>>> >>>     Hi Diego,
>>>> >>>     You may have to do some conversion as you have three fields
>in
>>>> >>> the
>>>> >>>     first line using the default space separator and five
>fields in
>>>> >>>     subsequent lines. If the first line doesn't contain any
>important
>>>> >>> data
>>>> >>>     you can just delete it or replace it with a meaningful
>header
>>>> >>> line
>>>> >>>     with five fields and save the file under another name.
>>>> >>>
>>>> >>>     It looks as thought you have date-time as two fields. If
>so, you
>>>> >>> can
>>>> >>>     just read the first field if you only want the date:
>>>> >>>
>>>> >>>     # assume you have removed the first line
>>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>>>> >>>
>>>> >>>     If you want the date/time:
>>>> >>>
>>>> >>>    
>dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>>>> >>> %H:%M:%S")
>>>> >>>
>>>> >>>     Jim
>>>> >>>
>>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
>>>> >>> <[hidden email]> wrote:
>>>> >>>     > Dear all,
>>>> >>>     >
>>>> >>>     > I am dealing with the reading of a *.txt file.
>>>> >>>     > The txt file the following shape:
>>>> >>>     >
>>>> >>>     > 103001930 103001580 103001530
>>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
>>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
>>>> >>>     > 1998-10-01 04:00:00 0 0 0
>>>> >>>     > 1998-10-01 05:00:00 0 0 0
>>>> >>>     > 1998-10-01 06:00:00 0 0 0
>>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
>>>> >>>     >
>>>> >>>     > If it is possible I have a coupe of questions, which will
>sound
>>>> >>> stupid but
>>>> >>>     > they are important to me in order to understand ho R deal
>with
>>>> >>> file
>>>> >>> or date.
>>>> >>>     >
>>>> >>>     > 1) Do I have to convert it to a *csv file?
>>>> >>>     > 2) Can a deal with space and not ","
>>>> >>>     > 3) How can I read date?
>>>> >>>     >
>>>> >>>     > thanks a lot to all of you,
>>>> >>>     > Thanks
>>>> >>>     >
>>>> >>>     >
>>>> >>>     > Diego
>>>> >>>     >
>>>> >>>     >         [[alternative HTML version deleted]]
>>>> >>>     >
>>>> >>>     > ______________________________________________
>>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and
>more,
>>>> >>> see
>>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
>>>> >>>     > PLEASE do read the posting guide
>>>> >>> http://www.R-project.org/posting-guide.html
>>>> >>>     > and provide commented, minimal, self-contained,
>reproducible
>>>> >>> code.
>>>> >>>
>>>> >>>     ______________________________________________
>>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and
>more, see
>>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
>>>> >>>     PLEASE do read the posting guide
>>>> >>> http://www.R-project.org/posting-guide.html
>>>> >>>     and provide commented, minimal, self-contained,
>reproducible
>>>> >>> code.
>>>> >>>
>>>> >>>
>>>> >>
>>>> >
>>>
>>>
>>
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear all,
I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat
all again in order to understand.
If I could I would like to start again, without mixing strategy and waiting
for your advice.

I am really appreciate you help, really really.
Here my new file, a *.csv file (buy the way, it is possible to attach it in
the mailing list?)

date,str1,str2,str3
10/1/1998 0:00,0.6,0,0
10/1/1998 1:00,0.2,0.2,0.2
10/1/1998 2:00,0.6,0.2,0.4
10/1/1998 3:00,0,0,0.6
10/1/1998 4:00,0,0,0
10/1/1998 5:00,0,0,0
10/1/1998 6:00,0,0,0
10/1/1998 7:00,0.2,0,0


I read it as:
MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")

at this point I would like to have the daily mean.
What would you suggest?

Really Really thanks,
You are my lifesaver

Thanks



Diego


On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]> wrote:

> ... and the most common source of NA values in time data is wrong
> timezones. You really need to make sure the timezone that is assumed when
> the character data are converted to POSIXt agrees with the data. In most
> cases the easiest way to insure this is to use
>
> Sys.setenv(TZ="US/Pacific")
>
> or whatever timezone from
>
> OlsonNames()
>
> corresponds with your data. Execute this setenv function before the
> strptime or as.POSIXct() function call.
>
> You can use
>
> MyData[ is.na(MyData$datetime), ]
>
> to see which records are failing to convert time.
>
> [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
>
> On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]> wrote:
> >Hi Diego,
> >I think the error is due to NA values in your data file. If I extend
> >your example and run it, I get no errors:
> >
> >MyData<-read.table(text="103001930 103001580 103001530
> >1998-10-01 00:00:00 0.6 0 0
> >1998-10-01 01:00:00 0.2 0.2 0.2
> >1998-10-01 02:00:00 0.6 0.2 0.4
> >1998-10-01 03:00:00 0 0 0.6
> >1998-10-01 04:00:00 0 0 0
> >1998-10-01 05:00:00 0 0 0
> >1998-10-01 06:00:00 0 0 0
> >1998-10-01 07:00:00 0.2 0 0
> >1998-10-01 08:00:00 0.6 0 0
> >1998-10-01 09:00:00 0.2 0.2 0.2
> >1998-10-01 10:00:00 0.6 0.2 0.4
> >1998-10-01 11:00:00 0 0 0.6
> >1998-10-01 12:00:00 0 0 0
> >1998-10-01 13:00:00 0 0 0
> >1998-10-01 14:00:00 0 0 0
> >1998-10-01 15:00:00 0.2 0 0
> >1998-10-01 16:00:00 0.6 0 0
> >1998-10-01 17:00:00 0.2 0.2 0.2
> >1998-10-01 18:00:00 0.6 0.2 0.4
> >1998-10-01 19:00:00 0 0 0.6
> >1998-10-01 20:00:00 0 0 0
> >1998-10-01 21:00:00 0 0 0
> >1998-10-01 22:00:00 0 0 0
> >1998-10-01 23:00:00 0.2 0 0
> >1998-10-02 00:00:00 0.6 0 0
> >1998-10-02 01:00:00 0.2 0.2 0.2
> >1998-10-02 02:00:00 0.6 0.2 0.4
> >1998-10-02 03:00:00 0 0 0.6
> >1998-10-02 04:00:00 0 0 0
> >1998-10-02 05:00:00 0 0 0
> >1998-10-02 06:00:00 0 0 0
> >1998-10-02 07:00:00 0.2 0 0
> >1998-10-02 08:00:00 0.6 0 0
> >1998-10-02 09:00:00 0.2 0.2 0.2
> >1998-10-02 10:00:00 0.6 0.2 0.4
> >1998-10-02 11:00:00 0 0 0.6
> >1998-10-02 12:00:00 0 0 0
> >1998-10-02 13:00:00 0 0 0
> >1998-10-02 14:00:00 0 0 0
> >1998-10-02 15:00:00 0.2 0 0
> >1998-10-02 16:00:00 0.6 0 0
> >1998-10-02 17:00:00 0.2 0.2 0.2
> >1998-10-02 18:00:00 0.6 0.2 0.4
> >1998-10-02 19:00:00 0 0 0.6
> >1998-10-02 20:00:00 0 0 0
> >1998-10-02 21:00:00 0 0 0
> >1998-10-02 22:00:00 0 0 0
> >1998-10-02 23:00:00 0.2 0 0",
> >skip=1,stringsAsFactors=FALSE)
> >names(MyData)<-c("date","time","st1","st2","st3")
> >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > format="%Y-%m-%d %H:%M:%S")
> >MyData$datetime
> >st1_daily<-by(MyData$st1,MyData$date,mean)
> >st2_daily<-by(MyData$st2,MyData$date,mean)
> >st3_daily<-by(MyData$st3,MyData$date,mean)
> >st1_daily
> >st2_daily
> >st3_daily
> >
> >Try adding na.rm=TRUE to the "by" calls:
> >
> >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
> >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
> >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
> >
> >Jim
> >
> >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
> ><[hidden email]> wrote:
> >> Dear all,
> >>
> >> I have still problem with date.
> >> Could you please tel me how to use POSIXct.
> >> Indeed I have found this command:
> >> timeAverage, but I am not able to convert MyDate to properly date.
> >>
> >> Thank a lot
> >> I hope to no bother you, at least too much
> >>
> >>
> >> Diego
> >>
> >>
> >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]>
> >wrote:
> >>>
> >>> Dear Jim, Dear all,
> >>>
> >>> thanks a lot.
> >>>
> >>> Unfortunately, I get the following error:
> >>>
> >>>
> >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
> >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
> >925L,  :
> >>>   arguments must have same length
> >>>
> >>>
> >>> This is particularly strange. indeed, if I apply
> >>>
> >>>
> >>> mean(MyData$str1,na.rm=TRUE)
> >>>
> >>>
> >>> it works
> >>>
> >>>
> >>> Sorry, I have to learn a lot.
> >>> You are really boosting me
> >>>
> >>> Diego
> >>>
> >>>
> >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
> >>>>
> >>>> Hi Diego,
> >>>> One way you can get daily means is:
> >>>>
> >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
> >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
> >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
> >>>>
> >>>> Jim
> >>>>
> >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
> ><[hidden email]>
> >>>> wrote:
> >>>> > Dear all,
> >>>> > I have found the error, my fault. Sorry.
> >>>> > There was an extra come in the headers line.
> >>>> > Thanks again.
> >>>> >
> >>>> > If I can I would like to ask you another questions about the
> >imported
> >>>> > data.
> >>>> > I would like to compute the daily average of the different date.
> >>>> > Basically I
> >>>> > have hourly data, I would like to ave the daily mean of them.
> >>>> >
> >>>> > Is there some special commands?
> >>>> >
> >>>> > Thanks a lot.
> >>>> >
> >>>> >
> >>>> > Diego
> >>>> >
> >>>> >
> >>>> > On 31 July 2018 at 10:40, Diego Avesani <[hidden email]>
> >>>> > wrote:
> >>>> >>
> >>>> >> Dear all,
> >>>> >> I move to csv file because originally the date where in csv
> >file.
> >>>> >> In addition, due to the fact that, as you told me, read.csv is a
> >>>> >> special
> >>>> >> case of read.table, I prefer start to learn from the simplest
> >one.
> >>>> >> After that, I will try also the *.txt format.
> >>>> >>
> >>>> >> with read.csv, something strange happened:
> >>>> >>
> >>>> >> This us now the file:
> >>>> >>
> >>>> >> date,st1,st2,st3,
> >>>> >> 10/1/1998 0:00,0.6,0,0
> >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
> >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
> >>>> >> 10/1/1998 3:00,0,0,0.6
> >>>> >> 10/1/1998 4:00,0,0,0
> >>>> >> 10/1/1998 5:00,0,0,0
> >>>> >> 10/1/1998 6:00,0,0,0
> >>>> >> 10/1/1998 7:00,0.2,0,0
> >>>> >> 10/1/1998 8:00,0.6,0.2,0
> >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
> >>>> >> 10/1/1998 10:00,0,0.4,0.2
> >>>> >>
> >>>> >> When I apply:
> >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> >>>> >>
> >>>> >> this is the results:
> >>>> >>
> >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
> >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
> >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
> >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
> >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
> >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
> >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
> >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
> >>>> >>
> >>>> >> I do not understand why.
> >>>> >> Something wrong with date?
> >>>> >>
> >>>> >> really really thanks,
> >>>> >> I appreciate a lot all your helps.
> >>>> >>
> >>>> >> Diedro
> >>>> >>
> >>>> >>
> >>>> >> Diego
> >>>> >>
> >>>> >>
> >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]>
> >wrote:
> >>>> >>>
> >>>> >>> Or, without removing the first line
> >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
> >>>> >>>
> >>>> >>> Another alternative,
> >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> >>>> >>> since the dates appear to be in the default format.
> >>>> >>> (I generally prefer to work with datetimes in POSIXct class
> >rather
> >>>> >>> than
> >>>> >>> POSIXlt class)
> >>>> >>>
> >>>> >>> -Don
> >>>> >>>
> >>>> >>> --
> >>>> >>> Don MacQueen
> >>>> >>> Lawrence Livermore National Laboratory
> >>>> >>> 7000 East Ave., L-627
> >>>> >>> Livermore, CA 94550
> >>>> >>> 925-423-1062
> >>>> >>> Lab cell 925-724-7509
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
> >>>> >>> <[hidden email] on behalf of
> >[hidden email]>
> >>>> >>> wrote:
> >>>> >>>
> >>>> >>>     Hi Diego,
> >>>> >>>     You may have to do some conversion as you have three fields
> >in
> >>>> >>> the
> >>>> >>>     first line using the default space separator and five
> >fields in
> >>>> >>>     subsequent lines. If the first line doesn't contain any
> >important
> >>>> >>> data
> >>>> >>>     you can just delete it or replace it with a meaningful
> >header
> >>>> >>> line
> >>>> >>>     with five fields and save the file under another name.
> >>>> >>>
> >>>> >>>     It looks as thought you have date-time as two fields. If
> >so, you
> >>>> >>> can
> >>>> >>>     just read the first field if you only want the date:
> >>>> >>>
> >>>> >>>     # assume you have removed the first line
> >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
> >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
> >>>> >>>
> >>>> >>>     If you want the date/time:
> >>>> >>>
> >>>> >>>
> >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> >>>> >>> %H:%M:%S")
> >>>> >>>
> >>>> >>>     Jim
> >>>> >>>
> >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
> >>>> >>> <[hidden email]> wrote:
> >>>> >>>     > Dear all,
> >>>> >>>     >
> >>>> >>>     > I am dealing with the reading of a *.txt file.
> >>>> >>>     > The txt file the following shape:
> >>>> >>>     >
> >>>> >>>     > 103001930 103001580 103001530
> >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
> >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
> >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
> >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
> >>>> >>>     > 1998-10-01 04:00:00 0 0 0
> >>>> >>>     > 1998-10-01 05:00:00 0 0 0
> >>>> >>>     > 1998-10-01 06:00:00 0 0 0
> >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
> >>>> >>>     >
> >>>> >>>     > If it is possible I have a coupe of questions, which will
> >sound
> >>>> >>> stupid but
> >>>> >>>     > they are important to me in order to understand ho R deal
> >with
> >>>> >>> file
> >>>> >>> or date.
> >>>> >>>     >
> >>>> >>>     > 1) Do I have to convert it to a *csv file?
> >>>> >>>     > 2) Can a deal with space and not ","
> >>>> >>>     > 3) How can I read date?
> >>>> >>>     >
> >>>> >>>     > thanks a lot to all of you,
> >>>> >>>     > Thanks
> >>>> >>>     >
> >>>> >>>     >
> >>>> >>>     > Diego
> >>>> >>>     >
> >>>> >>>     >         [[alternative HTML version deleted]]
> >>>> >>>     >
> >>>> >>>     > ______________________________________________
> >>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and
> >more,
> >>>> >>> see
> >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
> >>>> >>>     > PLEASE do read the posting guide
> >>>> >>> http://www.R-project.org/posting-guide.html
> >>>> >>>     > and provide commented, minimal, self-contained,
> >reproducible
> >>>> >>> code.
> >>>> >>>
> >>>> >>>     ______________________________________________
> >>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and
> >more, see
> >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
> >>>> >>>     PLEASE do read the posting guide
> >>>> >>> http://www.R-project.org/posting-guide.html
> >>>> >>>     and provide commented, minimal, self-contained,
> >reproducible
> >>>> >>> code.
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >
> >>>
> >>>
> >>
> >
> >______________________________________________
> >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> --
> Sent from my phone. Please excuse my brevity.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

PIKAL Petr
Hi

I did not get through all answers you already got and you probably obtained similar advice as mine.

# read data (if you have csv file just use read.csv)
> test<-read.table("clipboard", header=T, sep=",")

# control your object(s)
> str(test)
'data.frame':   8 obs. of  4 variables:
 $ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
 $ str1: num  0.6 0.2 0.6 0 0 0 0 0.2
 $ str2: num  0 0.2 0.2 0 0 0 0 0
 $ str3: num  0 0.2 0.4 0.6 0 0 0 0

#if it is OK change first column to real date by POSIXct
test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")

#attach lubridate
> library(lubridate)

# aggregate your object(s) and use lubridate function

> aggregate(test[,-1], list(day(test$date)), mean)
  Group.1 str1 str2 str3
1      10  0.2 0.05 0.15

# or format function

> aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
     Group.1 str1 str2 str3
1 1998-01-10  0.2 0.05 0.15

If it does not work with your data you should post at least result of

str(yourdata)

or preferably

dput(yourdata[1:20,])

Cheers
Petr


> -----Original Message-----
> From: R-help <[hidden email]> On Behalf Of Diego Avesani
> Sent: Wednesday, August 1, 2018 8:55 AM
> To: Jeff Newmiller <[hidden email]>
> Cc: r-help mailing list <[hidden email]>
> Subject: Re: [R] read txt file - date - no space
>
> Dear all,
> I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat all again
> in order to understand.
> If I could I would like to start again, without mixing strategy and waiting for
> your advice.
>
> I am really appreciate you help, really really.
> Here my new file, a *.csv file (buy the way, it is possible to attach it in the
> mailing list?)
>
> date,str1,str2,str3
> 10/1/1998 0:00,0.6,0,0
> 10/1/1998 1:00,0.2,0.2,0.2
> 10/1/1998 2:00,0.6,0.2,0.4
> 10/1/1998 3:00,0,0,0.6
> 10/1/1998 4:00,0,0,0
> 10/1/1998 5:00,0,0,0
> 10/1/1998 6:00,0,0,0
> 10/1/1998 7:00,0.2,0,0
>
>
> I read it as:
> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>
> at this point I would like to have the daily mean.
> What would you suggest?
>
> Really Really thanks,
> You are my lifesaver
>
> Thanks
>
>
>
> Diego
>
>
> On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]> wrote:
>
> > ... and the most common source of NA values in time data is wrong
> > timezones. You really need to make sure the timezone that is assumed
> > when the character data are converted to POSIXt agrees with the data.
> > In most cases the easiest way to insure this is to use
> >
> > Sys.setenv(TZ="US/Pacific")
> >
> > or whatever timezone from
> >
> > OlsonNames()
> >
> > corresponds with your data. Execute this setenv function before the
> > strptime or as.POSIXct() function call.
> >
> > You can use
> >
> > MyData[ is.na(MyData$datetime), ]
> >
> > to see which records are failing to convert time.
> >
> > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
> >
> > On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]>
> wrote:
> > >Hi Diego,
> > >I think the error is due to NA values in your data file. If I extend
> > >your example and run it, I get no errors:
> > >
> > >MyData<-read.table(text="103001930 103001580 103001530
> > >1998-10-01 00:00:00 0.6 0 0
> > >1998-10-01 01:00:00 0.2 0.2 0.2
> > >1998-10-01 02:00:00 0.6 0.2 0.4
> > >1998-10-01 03:00:00 0 0 0.6
> > >1998-10-01 04:00:00 0 0 0
> > >1998-10-01 05:00:00 0 0 0
> > >1998-10-01 06:00:00 0 0 0
> > >1998-10-01 07:00:00 0.2 0 0
> > >1998-10-01 08:00:00 0.6 0 0
> > >1998-10-01 09:00:00 0.2 0.2 0.2
> > >1998-10-01 10:00:00 0.6 0.2 0.4
> > >1998-10-01 11:00:00 0 0 0.6
> > >1998-10-01 12:00:00 0 0 0
> > >1998-10-01 13:00:00 0 0 0
> > >1998-10-01 14:00:00 0 0 0
> > >1998-10-01 15:00:00 0.2 0 0
> > >1998-10-01 16:00:00 0.6 0 0
> > >1998-10-01 17:00:00 0.2 0.2 0.2
> > >1998-10-01 18:00:00 0.6 0.2 0.4
> > >1998-10-01 19:00:00 0 0 0.6
> > >1998-10-01 20:00:00 0 0 0
> > >1998-10-01 21:00:00 0 0 0
> > >1998-10-01 22:00:00 0 0 0
> > >1998-10-01 23:00:00 0.2 0 0
> > >1998-10-02 00:00:00 0.6 0 0
> > >1998-10-02 01:00:00 0.2 0.2 0.2
> > >1998-10-02 02:00:00 0.6 0.2 0.4
> > >1998-10-02 03:00:00 0 0 0.6
> > >1998-10-02 04:00:00 0 0 0
> > >1998-10-02 05:00:00 0 0 0
> > >1998-10-02 06:00:00 0 0 0
> > >1998-10-02 07:00:00 0.2 0 0
> > >1998-10-02 08:00:00 0.6 0 0
> > >1998-10-02 09:00:00 0.2 0.2 0.2
> > >1998-10-02 10:00:00 0.6 0.2 0.4
> > >1998-10-02 11:00:00 0 0 0.6
> > >1998-10-02 12:00:00 0 0 0
> > >1998-10-02 13:00:00 0 0 0
> > >1998-10-02 14:00:00 0 0 0
> > >1998-10-02 15:00:00 0.2 0 0
> > >1998-10-02 16:00:00 0.6 0 0
> > >1998-10-02 17:00:00 0.2 0.2 0.2
> > >1998-10-02 18:00:00 0.6 0.2 0.4
> > >1998-10-02 19:00:00 0 0 0.6
> > >1998-10-02 20:00:00 0 0 0
> > >1998-10-02 21:00:00 0 0 0
> > >1998-10-02 22:00:00 0 0 0
> > >1998-10-02 23:00:00 0.2 0 0",
> > >skip=1,stringsAsFactors=FALSE)
> > >names(MyData)<-c("date","time","st1","st2","st3")
> > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > > format="%Y-%m-%d %H:%M:%S")
> > >MyData$datetime
> > >st1_daily<-by(MyData$st1,MyData$date,mean)
> > >st2_daily<-by(MyData$st2,MyData$date,mean)
> > >st3_daily<-by(MyData$st3,MyData$date,mean)
> > >st1_daily
> > >st2_daily
> > >st3_daily
> > >
> > >Try adding na.rm=TRUE to the "by" calls:
> > >
> > >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
> > >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
> > >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
> > >
> > >Jim
> > >
> > >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
> > ><[hidden email]> wrote:
> > >> Dear all,
> > >>
> > >> I have still problem with date.
> > >> Could you please tel me how to use POSIXct.
> > >> Indeed I have found this command:
> > >> timeAverage, but I am not able to convert MyDate to properly date.
> > >>
> > >> Thank a lot
> > >> I hope to no bother you, at least too much
> > >>
> > >>
> > >> Diego
> > >>
> > >>
> > >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]>
> > >wrote:
> > >>>
> > >>> Dear Jim, Dear all,
> > >>>
> > >>> thanks a lot.
> > >>>
> > >>> Unfortunately, I get the following error:
> > >>>
> > >>>
> > >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
> > >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
> > >925L,  :
> > >>>   arguments must have same length
> > >>>
> > >>>
> > >>> This is particularly strange. indeed, if I apply
> > >>>
> > >>>
> > >>> mean(MyData$str1,na.rm=TRUE)
> > >>>
> > >>>
> > >>> it works
> > >>>
> > >>>
> > >>> Sorry, I have to learn a lot.
> > >>> You are really boosting me
> > >>>
> > >>> Diego
> > >>>
> > >>>
> > >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
> > >>>>
> > >>>> Hi Diego,
> > >>>> One way you can get daily means is:
> > >>>>
> > >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
> > >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
> > >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
> > >>>>
> > >>>> Jim
> > >>>>
> > >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
> > ><[hidden email]>
> > >>>> wrote:
> > >>>> > Dear all,
> > >>>> > I have found the error, my fault. Sorry.
> > >>>> > There was an extra come in the headers line.
> > >>>> > Thanks again.
> > >>>> >
> > >>>> > If I can I would like to ask you another questions about the
> > >imported
> > >>>> > data.
> > >>>> > I would like to compute the daily average of the different date.
> > >>>> > Basically I
> > >>>> > have hourly data, I would like to ave the daily mean of them.
> > >>>> >
> > >>>> > Is there some special commands?
> > >>>> >
> > >>>> > Thanks a lot.
> > >>>> >
> > >>>> >
> > >>>> > Diego
> > >>>> >
> > >>>> >
> > >>>> > On 31 July 2018 at 10:40, Diego Avesani
> > >>>> > <[hidden email]>
> > >>>> > wrote:
> > >>>> >>
> > >>>> >> Dear all,
> > >>>> >> I move to csv file because originally the date where in csv
> > >file.
> > >>>> >> In addition, due to the fact that, as you told me, read.csv is
> > >>>> >> a special case of read.table, I prefer start to learn from the
> > >>>> >> simplest
> > >one.
> > >>>> >> After that, I will try also the *.txt format.
> > >>>> >>
> > >>>> >> with read.csv, something strange happened:
> > >>>> >>
> > >>>> >> This us now the file:
> > >>>> >>
> > >>>> >> date,st1,st2,st3,
> > >>>> >> 10/1/1998 0:00,0.6,0,0
> > >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
> > >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
> > >>>> >> 10/1/1998 3:00,0,0,0.6
> > >>>> >> 10/1/1998 4:00,0,0,0
> > >>>> >> 10/1/1998 5:00,0,0,0
> > >>>> >> 10/1/1998 6:00,0,0,0
> > >>>> >> 10/1/1998 7:00,0.2,0,0
> > >>>> >> 10/1/1998 8:00,0.6,0.2,0
> > >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
> > >>>> >> 10/1/1998 10:00,0,0.4,0.2
> > >>>> >>
> > >>>> >> When I apply:
> > >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> > >>>> >>
> > >>>> >> this is the results:
> > >>>> >>
> > >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
> > >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
> > >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
> > >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
> > >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
> > >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
> > >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
> > >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
> > >>>> >>
> > >>>> >> I do not understand why.
> > >>>> >> Something wrong with date?
> > >>>> >>
> > >>>> >> really really thanks,
> > >>>> >> I appreciate a lot all your helps.
> > >>>> >>
> > >>>> >> Diedro
> > >>>> >>
> > >>>> >>
> > >>>> >> Diego
> > >>>> >>
> > >>>> >>
> > >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]>
> > >wrote:
> > >>>> >>>
> > >>>> >>> Or, without removing the first line
> > >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE,
> > >>>> >>> skip=1)
> > >>>> >>>
> > >>>> >>> Another alternative,
> > >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> > >>>> >>> since the dates appear to be in the default format.
> > >>>> >>> (I generally prefer to work with datetimes in POSIXct class
> > >rather
> > >>>> >>> than
> > >>>> >>> POSIXlt class)
> > >>>> >>>
> > >>>> >>> -Don
> > >>>> >>>
> > >>>> >>> --
> > >>>> >>> Don MacQueen
> > >>>> >>> Lawrence Livermore National Laboratory
> > >>>> >>> 7000 East Ave., L-627
> > >>>> >>> Livermore, CA 94550
> > >>>> >>> 925-423-1062
> > >>>> >>> Lab cell 925-724-7509
> > >>>> >>>
> > >>>> >>>
> > >>>> >>>
> > >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
> > >>>> >>> <[hidden email] on behalf of
> > >[hidden email]>
> > >>>> >>> wrote:
> > >>>> >>>
> > >>>> >>>     Hi Diego,
> > >>>> >>>     You may have to do some conversion as you have three
> > >>>> >>> fields
> > >in
> > >>>> >>> the
> > >>>> >>>     first line using the default space separator and five
> > >fields in
> > >>>> >>>     subsequent lines. If the first line doesn't contain any
> > >important
> > >>>> >>> data
> > >>>> >>>     you can just delete it or replace it with a meaningful
> > >header
> > >>>> >>> line
> > >>>> >>>     with five fields and save the file under another name.
> > >>>> >>>
> > >>>> >>>     It looks as thought you have date-time as two fields. If
> > >so, you
> > >>>> >>> can
> > >>>> >>>     just read the first field if you only want the date:
> > >>>> >>>
> > >>>> >>>     # assume you have removed the first line
> > >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
> > >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
> > >>>> >>>
> > >>>> >>>     If you want the date/time:
> > >>>> >>>
> > >>>> >>>
> > >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> > >>>> >>> %H:%M:%S")
> > >>>> >>>
> > >>>> >>>     Jim
> > >>>> >>>
> > >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
> > >>>> >>> <[hidden email]> wrote:
> > >>>> >>>     > Dear all,
> > >>>> >>>     >
> > >>>> >>>     > I am dealing with the reading of a *.txt file.
> > >>>> >>>     > The txt file the following shape:
> > >>>> >>>     >
> > >>>> >>>     > 103001930 103001580 103001530
> > >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
> > >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
> > >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
> > >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
> > >>>> >>>     > 1998-10-01 04:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 05:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 06:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
> > >>>> >>>     >
> > >>>> >>>     > If it is possible I have a coupe of questions, which
> > >>>> >>> will
> > >sound
> > >>>> >>> stupid but
> > >>>> >>>     > they are important to me in order to understand ho R
> > >>>> >>> deal
> > >with
> > >>>> >>> file
> > >>>> >>> or date.
> > >>>> >>>     >
> > >>>> >>>     > 1) Do I have to convert it to a *csv file?
> > >>>> >>>     > 2) Can a deal with space and not ","
> > >>>> >>>     > 3) How can I read date?
> > >>>> >>>     >
> > >>>> >>>     > thanks a lot to all of you,
> > >>>> >>>     > Thanks
> > >>>> >>>     >
> > >>>> >>>     >
> > >>>> >>>     > Diego
> > >>>> >>>     >
> > >>>> >>>     >         [[alternative HTML version deleted]]
> > >>>> >>>     >
> > >>>> >>>     > ______________________________________________
> > >>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and
> > >more,
> > >>>> >>> see
> > >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
> > >>>> >>>     > PLEASE do read the posting guide
> > >>>> >>> http://www.R-project.org/posting-guide.html
> > >>>> >>>     > and provide commented, minimal, self-contained,
> > >reproducible
> > >>>> >>> code.
> > >>>> >>>
> > >>>> >>>     ______________________________________________
> > >>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and
> > >more, see
> > >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
> > >>>> >>>     PLEASE do read the posting guide
> > >>>> >>> http://www.R-project.org/posting-guide.html
> > >>>> >>>     and provide commented, minimal, self-contained,
> > >reproducible
> > >>>> >>> code.
> > >>>> >>>
> > >>>> >>>
> > >>>> >>
> > >>>> >
> > >>>
> > >>>
> > >>
> > >
> > >______________________________________________
> > >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > >https://stat.ethz.ch/mailman/listinfo/r-help
> > >PLEASE do read the posting guide
> > >http://www.R-project.org/posting-guide.html
> > >and provide commented, minimal, self-contained, reproducible code.
> >
> > --
> > Sent from my phone. Please excuse my brevity.
> >
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
Osobní údaje: Informace o zpracování a ochraně osobních údajů obchodních partnerů PRECHEZA a.s. jsou zveřejněny na: https://www.precheza.cz/zasady-ochrany-osobnich-udaju/ | Information about processing and protection of business partner’s personal data are available on website: https://www.precheza.cz/en/personal-data-protection-principles/
Důvěrnost: Tento e-mail a jakékoliv k němu připojené dokumenty jsou důvěrné a podléhají tomuto právně závaznému prohláąení o vyloučení odpovědnosti: https://www.precheza.cz/01-dovetek/ | This email and any documents attached to it may be confidential and are subject to the legally binding disclaimer: https://www.precheza.cz/en/01-disclaimer/

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear Pikal, Deal all,

again really thank.

it seems not working.
Some specifications: My non data are -999, but I could change it.

My final results is:

1        1  -55.86242 -55.84764660 -277.4775
2        2  -55.47554 -94.58921682 -277.4845
3        3  -55.47095 -99.20239198 -277.4709
4        4  -55.46470 -55.45952932 -392.9071
5        5  -55.43335 -55.40171682 -388.4110
6        6  -55.40108 -55.37399691 -332.9068
7        7  -55.39201 -55.35156250 -332.8902
8        8 -110.87184   0.16136188 -281.8230
9        9 -110.95077 -55.63856096 -332.9564
10      10 -157.64430  -0.06602705 -315.3840
11      11 -105.06157   0.11507675 -315.4152
12      12  -70.08677 -52.54501096 -316.7247


So it is not correct.
For example for the first day in my csv I would have expected 0.167.

I am going to post what you have suggested:

for  str(MyData)

'data.frame': 160008 obs. of  4 variables:
 $ date: POSIXct, format: "1998-01-10 00:00:00" "1998-01-10 01:00:00"
"1998-01-10 02:00:00" "1998-01-10 03:00:00" ...
 $ str1: num  0.6 0.2 0.6 0 0 0 0 0.2 0.6 0.2 ...
 $ str2: num  0 0.2 0.2 0 0 0 0 0 0.2 0.4 ...
 $ str3: num  0 0.2 0.4 0.6 0 0 0 0 0 0.4 ...

unfortunately, I am not able to post

dput(str[1:20,])

it gives me

Error in str[1:20, ] : object of type 'closure' is not subsettable


Thanks again,
I hope that what I posted could be enough in order to help me.



Diego


On 1 August 2018 at 13:58, PIKAL Petr <[hidden email]> wrote:

> Hi
>
> I did not get through all answers you already got and you probably
> obtained similar advice as mine.
>
> # read data (if you have csv file just use read.csv)
> > test<-read.table("clipboard", header=T, sep=",")
>
> # control your object(s)
> > str(test)
> 'data.frame':   8 obs. of  4 variables:
>  $ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
>  $ str1: num  0.6 0.2 0.6 0 0 0 0 0.2
>  $ str2: num  0 0.2 0.2 0 0 0 0 0
>  $ str3: num  0 0.2 0.4 0.6 0 0 0 0
>
> #if it is OK change first column to real date by POSIXct
> test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")
>
> #attach lubridate
> > library(lubridate)
>
> # aggregate your object(s) and use lubridate function
>
> > aggregate(test[,-1], list(day(test$date)), mean)
>   Group.1 str1 str2 str3
> 1      10  0.2 0.05 0.15
>
> # or format function
>
> > aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
>      Group.1 str1 str2 str3
> 1 1998-01-10  0.2 0.05 0.15
>
> If it does not work with your data you should post at least result of
>
> str(yourdata)
>
> or preferably
>
> dput(yourdata[1:20,])
>
> Cheers
> Petr
>
>
> > -----Original Message-----
> > From: R-help <[hidden email]> On Behalf Of Diego Avesani
> > Sent: Wednesday, August 1, 2018 8:55 AM
> > To: Jeff Newmiller <[hidden email]>
> > Cc: r-help mailing list <[hidden email]>
> > Subject: Re: [R] read txt file - date - no space
> >
> > Dear all,
> > I am sorry, I did a lot of confusion. I am sorry, I have to relax and
> stat all again
> > in order to understand.
> > If I could I would like to start again, without mixing strategy and
> waiting for
> > your advice.
> >
> > I am really appreciate you help, really really.
> > Here my new file, a *.csv file (buy the way, it is possible to attach it
> in the
> > mailing list?)
> >
> > date,str1,str2,str3
> > 10/1/1998 0:00,0.6,0,0
> > 10/1/1998 1:00,0.2,0.2,0.2
> > 10/1/1998 2:00,0.6,0.2,0.4
> > 10/1/1998 3:00,0,0,0.6
> > 10/1/1998 4:00,0,0,0
> > 10/1/1998 5:00,0,0,0
> > 10/1/1998 6:00,0,0,0
> > 10/1/1998 7:00,0.2,0,0
> >
> >
> > I read it as:
> > MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> >
> > at this point I would like to have the daily mean.
> > What would you suggest?
> >
> > Really Really thanks,
> > You are my lifesaver
> >
> > Thanks
> >
> >
> >
> > Diego
> >
> >
> > On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]>
> wrote:
> >
> > > ... and the most common source of NA values in time data is wrong
> > > timezones. You really need to make sure the timezone that is assumed
> > > when the character data are converted to POSIXt agrees with the data.
> > > In most cases the easiest way to insure this is to use
> > >
> > > Sys.setenv(TZ="US/Pacific")
> > >
> > > or whatever timezone from
> > >
> > > OlsonNames()
> > >
> > > corresponds with your data. Execute this setenv function before the
> > > strptime or as.POSIXct() function call.
> > >
> > > You can use
> > >
> > > MyData[ is.na(MyData$datetime), ]
> > >
> > > to see which records are failing to convert time.
> > >
> > > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
> > >
> > > On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]>
> > wrote:
> > > >Hi Diego,
> > > >I think the error is due to NA values in your data file. If I extend
> > > >your example and run it, I get no errors:
> > > >
> > > >MyData<-read.table(text="103001930 103001580 103001530
> > > >1998-10-01 00:00:00 0.6 0 0
> > > >1998-10-01 01:00:00 0.2 0.2 0.2
> > > >1998-10-01 02:00:00 0.6 0.2 0.4
> > > >1998-10-01 03:00:00 0 0 0.6
> > > >1998-10-01 04:00:00 0 0 0
> > > >1998-10-01 05:00:00 0 0 0
> > > >1998-10-01 06:00:00 0 0 0
> > > >1998-10-01 07:00:00 0.2 0 0
> > > >1998-10-01 08:00:00 0.6 0 0
> > > >1998-10-01 09:00:00 0.2 0.2 0.2
> > > >1998-10-01 10:00:00 0.6 0.2 0.4
> > > >1998-10-01 11:00:00 0 0 0.6
> > > >1998-10-01 12:00:00 0 0 0
> > > >1998-10-01 13:00:00 0 0 0
> > > >1998-10-01 14:00:00 0 0 0
> > > >1998-10-01 15:00:00 0.2 0 0
> > > >1998-10-01 16:00:00 0.6 0 0
> > > >1998-10-01 17:00:00 0.2 0.2 0.2
> > > >1998-10-01 18:00:00 0.6 0.2 0.4
> > > >1998-10-01 19:00:00 0 0 0.6
> > > >1998-10-01 20:00:00 0 0 0
> > > >1998-10-01 21:00:00 0 0 0
> > > >1998-10-01 22:00:00 0 0 0
> > > >1998-10-01 23:00:00 0.2 0 0
> > > >1998-10-02 00:00:00 0.6 0 0
> > > >1998-10-02 01:00:00 0.2 0.2 0.2
> > > >1998-10-02 02:00:00 0.6 0.2 0.4
> > > >1998-10-02 03:00:00 0 0 0.6
> > > >1998-10-02 04:00:00 0 0 0
> > > >1998-10-02 05:00:00 0 0 0
> > > >1998-10-02 06:00:00 0 0 0
> > > >1998-10-02 07:00:00 0.2 0 0
> > > >1998-10-02 08:00:00 0.6 0 0
> > > >1998-10-02 09:00:00 0.2 0.2 0.2
> > > >1998-10-02 10:00:00 0.6 0.2 0.4
> > > >1998-10-02 11:00:00 0 0 0.6
> > > >1998-10-02 12:00:00 0 0 0
> > > >1998-10-02 13:00:00 0 0 0
> > > >1998-10-02 14:00:00 0 0 0
> > > >1998-10-02 15:00:00 0.2 0 0
> > > >1998-10-02 16:00:00 0.6 0 0
> > > >1998-10-02 17:00:00 0.2 0.2 0.2
> > > >1998-10-02 18:00:00 0.6 0.2 0.4
> > > >1998-10-02 19:00:00 0 0 0.6
> > > >1998-10-02 20:00:00 0 0 0
> > > >1998-10-02 21:00:00 0 0 0
> > > >1998-10-02 22:00:00 0 0 0
> > > >1998-10-02 23:00:00 0.2 0 0",
> > > >skip=1,stringsAsFactors=FALSE)
> > > >names(MyData)<-c("date","time","st1","st2","st3")
> > > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > > > format="%Y-%m-%d %H:%M:%S")
> > > >MyData$datetime
> > > >st1_daily<-by(MyData$st1,MyData$date,mean)
> > > >st2_daily<-by(MyData$st2,MyData$date,mean)
> > > >st3_daily<-by(MyData$st3,MyData$date,mean)
> > > >st1_daily
> > > >st2_daily
> > > >st3_daily
> > > >
> > > >Try adding na.rm=TRUE to the "by" calls:
> > > >
> > > >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
> > > >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
> > > >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
> > > >
> > > >Jim
> > > >
> > > >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
> > > ><[hidden email]> wrote:
> > > >> Dear all,
> > > >>
> > > >> I have still problem with date.
> > > >> Could you please tel me how to use POSIXct.
> > > >> Indeed I have found this command:
> > > >> timeAverage, but I am not able to convert MyDate to properly date.
> > > >>
> > > >> Thank a lot
> > > >> I hope to no bother you, at least too much
> > > >>
> > > >>
> > > >> Diego
> > > >>
> > > >>
> > > >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]>
> > > >wrote:
> > > >>>
> > > >>> Dear Jim, Dear all,
> > > >>>
> > > >>> thanks a lot.
> > > >>>
> > > >>> Unfortunately, I get the following error:
> > > >>>
> > > >>>
> > > >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
> > > >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
> > > >925L,  :
> > > >>>   arguments must have same length
> > > >>>
> > > >>>
> > > >>> This is particularly strange. indeed, if I apply
> > > >>>
> > > >>>
> > > >>> mean(MyData$str1,na.rm=TRUE)
> > > >>>
> > > >>>
> > > >>> it works
> > > >>>
> > > >>>
> > > >>> Sorry, I have to learn a lot.
> > > >>> You are really boosting me
> > > >>>
> > > >>> Diego
> > > >>>
> > > >>>
> > > >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
> > > >>>>
> > > >>>> Hi Diego,
> > > >>>> One way you can get daily means is:
> > > >>>>
> > > >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
> > > >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
> > > >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
> > > >>>>
> > > >>>> Jim
> > > >>>>
> > > >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
> > > ><[hidden email]>
> > > >>>> wrote:
> > > >>>> > Dear all,
> > > >>>> > I have found the error, my fault. Sorry.
> > > >>>> > There was an extra come in the headers line.
> > > >>>> > Thanks again.
> > > >>>> >
> > > >>>> > If I can I would like to ask you another questions about the
> > > >imported
> > > >>>> > data.
> > > >>>> > I would like to compute the daily average of the different date.
> > > >>>> > Basically I
> > > >>>> > have hourly data, I would like to ave the daily mean of them.
> > > >>>> >
> > > >>>> > Is there some special commands?
> > > >>>> >
> > > >>>> > Thanks a lot.
> > > >>>> >
> > > >>>> >
> > > >>>> > Diego
> > > >>>> >
> > > >>>> >
> > > >>>> > On 31 July 2018 at 10:40, Diego Avesani
> > > >>>> > <[hidden email]>
> > > >>>> > wrote:
> > > >>>> >>
> > > >>>> >> Dear all,
> > > >>>> >> I move to csv file because originally the date where in csv
> > > >file.
> > > >>>> >> In addition, due to the fact that, as you told me, read.csv is
> > > >>>> >> a special case of read.table, I prefer start to learn from the
> > > >>>> >> simplest
> > > >one.
> > > >>>> >> After that, I will try also the *.txt format.
> > > >>>> >>
> > > >>>> >> with read.csv, something strange happened:
> > > >>>> >>
> > > >>>> >> This us now the file:
> > > >>>> >>
> > > >>>> >> date,st1,st2,st3,
> > > >>>> >> 10/1/1998 0:00,0.6,0,0
> > > >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
> > > >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
> > > >>>> >> 10/1/1998 3:00,0,0,0.6
> > > >>>> >> 10/1/1998 4:00,0,0,0
> > > >>>> >> 10/1/1998 5:00,0,0,0
> > > >>>> >> 10/1/1998 6:00,0,0,0
> > > >>>> >> 10/1/1998 7:00,0.2,0,0
> > > >>>> >> 10/1/1998 8:00,0.6,0.2,0
> > > >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
> > > >>>> >> 10/1/1998 10:00,0,0.4,0.2
> > > >>>> >>
> > > >>>> >> When I apply:
> > > >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> > > >>>> >>
> > > >>>> >> this is the results:
> > > >>>> >>
> > > >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
> > > >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
> > > >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
> > > >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
> > > >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
> > > >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
> > > >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
> > > >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
> > > >>>> >>
> > > >>>> >> I do not understand why.
> > > >>>> >> Something wrong with date?
> > > >>>> >>
> > > >>>> >> really really thanks,
> > > >>>> >> I appreciate a lot all your helps.
> > > >>>> >>
> > > >>>> >> Diedro
> > > >>>> >>
> > > >>>> >>
> > > >>>> >> Diego
> > > >>>> >>
> > > >>>> >>
> > > >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]>
> > > >wrote:
> > > >>>> >>>
> > > >>>> >>> Or, without removing the first line
> > > >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE,
> > > >>>> >>> skip=1)
> > > >>>> >>>
> > > >>>> >>> Another alternative,
> > > >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> > > >>>> >>> since the dates appear to be in the default format.
> > > >>>> >>> (I generally prefer to work with datetimes in POSIXct class
> > > >rather
> > > >>>> >>> than
> > > >>>> >>> POSIXlt class)
> > > >>>> >>>
> > > >>>> >>> -Don
> > > >>>> >>>
> > > >>>> >>> --
> > > >>>> >>> Don MacQueen
> > > >>>> >>> Lawrence Livermore National Laboratory
> > > >>>> >>> 7000 East Ave., L-627
> > > >>>> >>> Livermore, CA 94550
> > > >>>> >>> 925-423-1062
> > > >>>> >>> Lab cell 925-724-7509
> > > >>>> >>>
> > > >>>> >>>
> > > >>>> >>>
> > > >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
> > > >>>> >>> <[hidden email] on behalf of
> > > >[hidden email]>
> > > >>>> >>> wrote:
> > > >>>> >>>
> > > >>>> >>>     Hi Diego,
> > > >>>> >>>     You may have to do some conversion as you have three
> > > >>>> >>> fields
> > > >in
> > > >>>> >>> the
> > > >>>> >>>     first line using the default space separator and five
> > > >fields in
> > > >>>> >>>     subsequent lines. If the first line doesn't contain any
> > > >important
> > > >>>> >>> data
> > > >>>> >>>     you can just delete it or replace it with a meaningful
> > > >header
> > > >>>> >>> line
> > > >>>> >>>     with five fields and save the file under another name.
> > > >>>> >>>
> > > >>>> >>>     It looks as thought you have date-time as two fields. If
> > > >so, you
> > > >>>> >>> can
> > > >>>> >>>     just read the first field if you only want the date:
> > > >>>> >>>
> > > >>>> >>>     # assume you have removed the first line
> > > >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
> > > >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
> > > >>>> >>>
> > > >>>> >>>     If you want the date/time:
> > > >>>> >>>
> > > >>>> >>>
> > > >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> > > >>>> >>> %H:%M:%S")
> > > >>>> >>>
> > > >>>> >>>     Jim
> > > >>>> >>>
> > > >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
> > > >>>> >>> <[hidden email]> wrote:
> > > >>>> >>>     > Dear all,
> > > >>>> >>>     >
> > > >>>> >>>     > I am dealing with the reading of a *.txt file.
> > > >>>> >>>     > The txt file the following shape:
> > > >>>> >>>     >
> > > >>>> >>>     > 103001930 103001580 103001530
> > > >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
> > > >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
> > > >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
> > > >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
> > > >>>> >>>     > 1998-10-01 04:00:00 0 0 0
> > > >>>> >>>     > 1998-10-01 05:00:00 0 0 0
> > > >>>> >>>     > 1998-10-01 06:00:00 0 0 0
> > > >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
> > > >>>> >>>     >
> > > >>>> >>>     > If it is possible I have a coupe of questions, which
> > > >>>> >>> will
> > > >sound
> > > >>>> >>> stupid but
> > > >>>> >>>     > they are important to me in order to understand ho R
> > > >>>> >>> deal
> > > >with
> > > >>>> >>> file
> > > >>>> >>> or date.
> > > >>>> >>>     >
> > > >>>> >>>     > 1) Do I have to convert it to a *csv file?
> > > >>>> >>>     > 2) Can a deal with space and not ","
> > > >>>> >>>     > 3) How can I read date?
> > > >>>> >>>     >
> > > >>>> >>>     > thanks a lot to all of you,
> > > >>>> >>>     > Thanks
> > > >>>> >>>     >
> > > >>>> >>>     >
> > > >>>> >>>     > Diego
> > > >>>> >>>     >
> > > >>>> >>>     >         [[alternative HTML version deleted]]
> > > >>>> >>>     >
> > > >>>> >>>     > ______________________________________________
> > > >>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and
> > > >more,
> > > >>>> >>> see
> > > >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
> > > >>>> >>>     > PLEASE do read the posting guide
> > > >>>> >>> http://www.R-project.org/posting-guide.html
> > > >>>> >>>     > and provide commented, minimal, self-contained,
> > > >reproducible
> > > >>>> >>> code.
> > > >>>> >>>
> > > >>>> >>>     ______________________________________________
> > > >>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and
> > > >more, see
> > > >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
> > > >>>> >>>     PLEASE do read the posting guide
> > > >>>> >>> http://www.R-project.org/posting-guide.html
> > > >>>> >>>     and provide commented, minimal, self-contained,
> > > >reproducible
> > > >>>> >>> code.
> > > >>>> >>>
> > > >>>> >>>
> > > >>>> >>
> > > >>>> >
> > > >>>
> > > >>>
> > > >>
> > > >
> > > >______________________________________________
> > > >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > > >https://stat.ethz.ch/mailman/listinfo/r-help
> > > >PLEASE do read the posting guide
> > > >http://www.R-project.org/posting-guide.html
> > > >and provide commented, minimal, self-contained, reproducible code.
> > >
> > > --
> > > Sent from my phone. Please excuse my brevity.
> > >
> >
> > [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> Osobní údaje: Informace o zpracování a ochraně osobních údajů obchodních
> partnerů PRECHEZA a.s. jsou zveřejněny na: https://www.precheza.cz/
> zasady-ochrany-osobnich-udaju/ | Information about processing and
> protection of business partner’s personal data are available on website:
> https://www.precheza.cz/en/personal-data-protection-principles/
> Důvěrnost: Tento e-mail a jakékoliv k němu připojené dokumenty jsou
> důvěrné a podléhají tomuto právně závaznému prohláąení o vyloučení
> odpovědnosti: https://www.precheza.cz/01-dovetek/ | This email and any
> documents attached to it may be confidential and are subject to the legally
> binding disclaimer: https://www.precheza.cz/en/01-disclaimer/
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear Pikal, DEar all,

I do not if it could help:
if I print MyData%date, I get (at some point)

[281] "1998-12-10 16:00:00 CET"  "1998-12-10 17:00:00 CET"
"1998-12-10 18:00:00 CET"  "1998-12-10 19:00:00 CET"
 [285] "1998-12-10 20:00:00 CET"  "1998-12-10 21:00:00 CET"
"1998-12-10 22:00:00 CET"  "1998-12-10 23:00:00 CET"
 [289] NA                         NA                         NA
                 NA
 [293] NA                         NA                         NA
                 NA
 [297] NA                         NA                         NA
                 NA


Again Thanks

Diego


On 1 August 2018 at 14:29, Diego Avesani <[hidden email]> wrote:

> Dear Pikal, Deal all,
>
> again really thank.
>
> it seems not working.
> Some specifications: My non data are -999, but I could change it.
>
> My final results is:
>
> 1        1  -55.86242 -55.84764660 -277.4775
> 2        2  -55.47554 -94.58921682 -277.4845
> 3        3  -55.47095 -99.20239198 -277.4709
> 4        4  -55.46470 -55.45952932 -392.9071
> 5        5  -55.43335 -55.40171682 -388.4110
> 6        6  -55.40108 -55.37399691 -332.9068
> 7        7  -55.39201 -55.35156250 -332.8902
> 8        8 -110.87184   0.16136188 -281.8230
> 9        9 -110.95077 -55.63856096 -332.9564
> 10      10 -157.64430  -0.06602705 -315.3840
> 11      11 -105.06157   0.11507675 -315.4152
> 12      12  -70.08677 -52.54501096 -316.7247
>
>
> So it is not correct.
> For example for the first day in my csv I would have expected 0.167.
>
> I am going to post what you have suggested:
>
> for  str(MyData)
>
> 'data.frame': 160008 obs. of  4 variables:
>  $ date: POSIXct, format: "1998-01-10 00:00:00" "1998-01-10 01:00:00"
> "1998-01-10 02:00:00" "1998-01-10 03:00:00" ...
>  $ str1: num  0.6 0.2 0.6 0 0 0 0 0.2 0.6 0.2 ...
>  $ str2: num  0 0.2 0.2 0 0 0 0 0 0.2 0.4 ...
>  $ str3: num  0 0.2 0.4 0.6 0 0 0 0 0 0.4 ...
>
> unfortunately, I am not able to post
>
> dput(str[1:20,])
>
> it gives me
>
> Error in str[1:20, ] : object of type 'closure' is not subsettable
>
>
> Thanks again,
> I hope that what I posted could be enough in order to help me.
>
>
>
> Diego
>
>
> On 1 August 2018 at 13:58, PIKAL Petr <[hidden email]> wrote:
>
>> Hi
>>
>> I did not get through all answers you already got and you probably
>> obtained similar advice as mine.
>>
>> # read data (if you have csv file just use read.csv)
>> > test<-read.table("clipboard", header=T, sep=",")
>>
>> # control your object(s)
>> > str(test)
>> 'data.frame':   8 obs. of  4 variables:
>>  $ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
>>  $ str1: num  0.6 0.2 0.6 0 0 0 0 0.2
>>  $ str2: num  0 0.2 0.2 0 0 0 0 0
>>  $ str3: num  0 0.2 0.4 0.6 0 0 0 0
>>
>> #if it is OK change first column to real date by POSIXct
>> test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")
>>
>> #attach lubridate
>> > library(lubridate)
>>
>> # aggregate your object(s) and use lubridate function
>>
>> > aggregate(test[,-1], list(day(test$date)), mean)
>>   Group.1 str1 str2 str3
>> 1      10  0.2 0.05 0.15
>>
>> # or format function
>>
>> > aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
>>      Group.1 str1 str2 str3
>> 1 1998-01-10  0.2 0.05 0.15
>>
>> If it does not work with your data you should post at least result of
>>
>> str(yourdata)
>>
>> or preferably
>>
>> dput(yourdata[1:20,])
>>
>> Cheers
>> Petr
>>
>>
>> > -----Original Message-----
>> > From: R-help <[hidden email]> On Behalf Of Diego Avesani
>> > Sent: Wednesday, August 1, 2018 8:55 AM
>> > To: Jeff Newmiller <[hidden email]>
>> > Cc: r-help mailing list <[hidden email]>
>> > Subject: Re: [R] read txt file - date - no space
>> >
>> > Dear all,
>> > I am sorry, I did a lot of confusion. I am sorry, I have to relax and
>> stat all again
>> > in order to understand.
>> > If I could I would like to start again, without mixing strategy and
>> waiting for
>> > your advice.
>> >
>> > I am really appreciate you help, really really.
>> > Here my new file, a *.csv file (buy the way, it is possible to attach
>> it in the
>> > mailing list?)
>> >
>> > date,str1,str2,str3
>> > 10/1/1998 0:00,0.6,0,0
>> > 10/1/1998 1:00,0.2,0.2,0.2
>> > 10/1/1998 2:00,0.6,0.2,0.4
>> > 10/1/1998 3:00,0,0,0.6
>> > 10/1/1998 4:00,0,0,0
>> > 10/1/1998 5:00,0,0,0
>> > 10/1/1998 6:00,0,0,0
>> > 10/1/1998 7:00,0.2,0,0
>> >
>> >
>> > I read it as:
>> > MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>> >
>> > at this point I would like to have the daily mean.
>> > What would you suggest?
>> >
>> > Really Really thanks,
>> > You are my lifesaver
>> >
>> > Thanks
>> >
>> >
>> >
>> > Diego
>> >
>> >
>> > On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]>
>> wrote:
>> >
>> > > ... and the most common source of NA values in time data is wrong
>> > > timezones. You really need to make sure the timezone that is assumed
>> > > when the character data are converted to POSIXt agrees with the data.
>> > > In most cases the easiest way to insure this is to use
>> > >
>> > > Sys.setenv(TZ="US/Pacific")
>> > >
>> > > or whatever timezone from
>> > >
>> > > OlsonNames()
>> > >
>> > > corresponds with your data. Execute this setenv function before the
>> > > strptime or as.POSIXct() function call.
>> > >
>> > > You can use
>> > >
>> > > MyData[ is.na(MyData$datetime), ]
>> > >
>> > > to see which records are failing to convert time.
>> > >
>> > > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
>> > >
>> > > On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]>
>> > wrote:
>> > > >Hi Diego,
>> > > >I think the error is due to NA values in your data file. If I extend
>> > > >your example and run it, I get no errors:
>> > > >
>> > > >MyData<-read.table(text="103001930 103001580 103001530
>> > > >1998-10-01 00:00:00 0.6 0 0
>> > > >1998-10-01 01:00:00 0.2 0.2 0.2
>> > > >1998-10-01 02:00:00 0.6 0.2 0.4
>> > > >1998-10-01 03:00:00 0 0 0.6
>> > > >1998-10-01 04:00:00 0 0 0
>> > > >1998-10-01 05:00:00 0 0 0
>> > > >1998-10-01 06:00:00 0 0 0
>> > > >1998-10-01 07:00:00 0.2 0 0
>> > > >1998-10-01 08:00:00 0.6 0 0
>> > > >1998-10-01 09:00:00 0.2 0.2 0.2
>> > > >1998-10-01 10:00:00 0.6 0.2 0.4
>> > > >1998-10-01 11:00:00 0 0 0.6
>> > > >1998-10-01 12:00:00 0 0 0
>> > > >1998-10-01 13:00:00 0 0 0
>> > > >1998-10-01 14:00:00 0 0 0
>> > > >1998-10-01 15:00:00 0.2 0 0
>> > > >1998-10-01 16:00:00 0.6 0 0
>> > > >1998-10-01 17:00:00 0.2 0.2 0.2
>> > > >1998-10-01 18:00:00 0.6 0.2 0.4
>> > > >1998-10-01 19:00:00 0 0 0.6
>> > > >1998-10-01 20:00:00 0 0 0
>> > > >1998-10-01 21:00:00 0 0 0
>> > > >1998-10-01 22:00:00 0 0 0
>> > > >1998-10-01 23:00:00 0.2 0 0
>> > > >1998-10-02 00:00:00 0.6 0 0
>> > > >1998-10-02 01:00:00 0.2 0.2 0.2
>> > > >1998-10-02 02:00:00 0.6 0.2 0.4
>> > > >1998-10-02 03:00:00 0 0 0.6
>> > > >1998-10-02 04:00:00 0 0 0
>> > > >1998-10-02 05:00:00 0 0 0
>> > > >1998-10-02 06:00:00 0 0 0
>> > > >1998-10-02 07:00:00 0.2 0 0
>> > > >1998-10-02 08:00:00 0.6 0 0
>> > > >1998-10-02 09:00:00 0.2 0.2 0.2
>> > > >1998-10-02 10:00:00 0.6 0.2 0.4
>> > > >1998-10-02 11:00:00 0 0 0.6
>> > > >1998-10-02 12:00:00 0 0 0
>> > > >1998-10-02 13:00:00 0 0 0
>> > > >1998-10-02 14:00:00 0 0 0
>> > > >1998-10-02 15:00:00 0.2 0 0
>> > > >1998-10-02 16:00:00 0.6 0 0
>> > > >1998-10-02 17:00:00 0.2 0.2 0.2
>> > > >1998-10-02 18:00:00 0.6 0.2 0.4
>> > > >1998-10-02 19:00:00 0 0 0.6
>> > > >1998-10-02 20:00:00 0 0 0
>> > > >1998-10-02 21:00:00 0 0 0
>> > > >1998-10-02 22:00:00 0 0 0
>> > > >1998-10-02 23:00:00 0.2 0 0",
>> > > >skip=1,stringsAsFactors=FALSE)
>> > > >names(MyData)<-c("date","time","st1","st2","st3")
>> > > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
>> > > > format="%Y-%m-%d %H:%M:%S")
>> > > >MyData$datetime
>> > > >st1_daily<-by(MyData$st1,MyData$date,mean)
>> > > >st2_daily<-by(MyData$st2,MyData$date,mean)
>> > > >st3_daily<-by(MyData$st3,MyData$date,mean)
>> > > >st1_daily
>> > > >st2_daily
>> > > >st3_daily
>> > > >
>> > > >Try adding na.rm=TRUE to the "by" calls:
>> > > >
>> > > >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
>> > > >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
>> > > >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
>> > > >
>> > > >Jim
>> > > >
>> > > >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
>> > > ><[hidden email]> wrote:
>> > > >> Dear all,
>> > > >>
>> > > >> I have still problem with date.
>> > > >> Could you please tel me how to use POSIXct.
>> > > >> Indeed I have found this command:
>> > > >> timeAverage, but I am not able to convert MyDate to properly date.
>> > > >>
>> > > >> Thank a lot
>> > > >> I hope to no bother you, at least too much
>> > > >>
>> > > >>
>> > > >> Diego
>> > > >>
>> > > >>
>> > > >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]>
>> > > >wrote:
>> > > >>>
>> > > >>> Dear Jim, Dear all,
>> > > >>>
>> > > >>> thanks a lot.
>> > > >>>
>> > > >>> Unfortunately, I get the following error:
>> > > >>>
>> > > >>>
>> > > >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
>> > > >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
>> > > >925L,  :
>> > > >>>   arguments must have same length
>> > > >>>
>> > > >>>
>> > > >>> This is particularly strange. indeed, if I apply
>> > > >>>
>> > > >>>
>> > > >>> mean(MyData$str1,na.rm=TRUE)
>> > > >>>
>> > > >>>
>> > > >>> it works
>> > > >>>
>> > > >>>
>> > > >>> Sorry, I have to learn a lot.
>> > > >>> You are really boosting me
>> > > >>>
>> > > >>> Diego
>> > > >>>
>> > > >>>
>> > > >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
>> > > >>>>
>> > > >>>> Hi Diego,
>> > > >>>> One way you can get daily means is:
>> > > >>>>
>> > > >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
>> > > >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
>> > > >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
>> > > >>>>
>> > > >>>> Jim
>> > > >>>>
>> > > >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
>> > > ><[hidden email]>
>> > > >>>> wrote:
>> > > >>>> > Dear all,
>> > > >>>> > I have found the error, my fault. Sorry.
>> > > >>>> > There was an extra come in the headers line.
>> > > >>>> > Thanks again.
>> > > >>>> >
>> > > >>>> > If I can I would like to ask you another questions about the
>> > > >imported
>> > > >>>> > data.
>> > > >>>> > I would like to compute the daily average of the different
>> date.
>> > > >>>> > Basically I
>> > > >>>> > have hourly data, I would like to ave the daily mean of them.
>> > > >>>> >
>> > > >>>> > Is there some special commands?
>> > > >>>> >
>> > > >>>> > Thanks a lot.
>> > > >>>> >
>> > > >>>> >
>> > > >>>> > Diego
>> > > >>>> >
>> > > >>>> >
>> > > >>>> > On 31 July 2018 at 10:40, Diego Avesani
>> > > >>>> > <[hidden email]>
>> > > >>>> > wrote:
>> > > >>>> >>
>> > > >>>> >> Dear all,
>> > > >>>> >> I move to csv file because originally the date where in csv
>> > > >file.
>> > > >>>> >> In addition, due to the fact that, as you told me, read.csv is
>> > > >>>> >> a special case of read.table, I prefer start to learn from the
>> > > >>>> >> simplest
>> > > >one.
>> > > >>>> >> After that, I will try also the *.txt format.
>> > > >>>> >>
>> > > >>>> >> with read.csv, something strange happened:
>> > > >>>> >>
>> > > >>>> >> This us now the file:
>> > > >>>> >>
>> > > >>>> >> date,st1,st2,st3,
>> > > >>>> >> 10/1/1998 0:00,0.6,0,0
>> > > >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
>> > > >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
>> > > >>>> >> 10/1/1998 3:00,0,0,0.6
>> > > >>>> >> 10/1/1998 4:00,0,0,0
>> > > >>>> >> 10/1/1998 5:00,0,0,0
>> > > >>>> >> 10/1/1998 6:00,0,0,0
>> > > >>>> >> 10/1/1998 7:00,0.2,0,0
>> > > >>>> >> 10/1/1998 8:00,0.6,0.2,0
>> > > >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
>> > > >>>> >> 10/1/1998 10:00,0,0.4,0.2
>> > > >>>> >>
>> > > >>>> >> When I apply:
>> > > >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>> > > >>>> >>
>> > > >>>> >> this is the results:
>> > > >>>> >>
>> > > >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
>> > > >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
>> > > >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
>> > > >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
>> > > >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
>> > > >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
>> > > >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
>> > > >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
>> > > >>>> >>
>> > > >>>> >> I do not understand why.
>> > > >>>> >> Something wrong with date?
>> > > >>>> >>
>> > > >>>> >> really really thanks,
>> > > >>>> >> I appreciate a lot all your helps.
>> > > >>>> >>
>> > > >>>> >> Diedro
>> > > >>>> >>
>> > > >>>> >>
>> > > >>>> >> Diego
>> > > >>>> >>
>> > > >>>> >>
>> > > >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]>
>> > > >wrote:
>> > > >>>> >>>
>> > > >>>> >>> Or, without removing the first line
>> > > >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE,
>> > > >>>> >>> skip=1)
>> > > >>>> >>>
>> > > >>>> >>> Another alternative,
>> > > >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>> > > >>>> >>> since the dates appear to be in the default format.
>> > > >>>> >>> (I generally prefer to work with datetimes in POSIXct class
>> > > >rather
>> > > >>>> >>> than
>> > > >>>> >>> POSIXlt class)
>> > > >>>> >>>
>> > > >>>> >>> -Don
>> > > >>>> >>>
>> > > >>>> >>> --
>> > > >>>> >>> Don MacQueen
>> > > >>>> >>> Lawrence Livermore National Laboratory
>> > > >>>> >>> 7000 East Ave., L-627
>> > > >>>> >>> Livermore, CA 94550
>> > > >>>> >>> 925-423-1062
>> > > >>>> >>> Lab cell 925-724-7509
>> > > >>>> >>>
>> > > >>>> >>>
>> > > >>>> >>>
>> > > >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
>> > > >>>> >>> <[hidden email] on behalf of
>> > > >[hidden email]>
>> > > >>>> >>> wrote:
>> > > >>>> >>>
>> > > >>>> >>>     Hi Diego,
>> > > >>>> >>>     You may have to do some conversion as you have three
>> > > >>>> >>> fields
>> > > >in
>> > > >>>> >>> the
>> > > >>>> >>>     first line using the default space separator and five
>> > > >fields in
>> > > >>>> >>>     subsequent lines. If the first line doesn't contain any
>> > > >important
>> > > >>>> >>> data
>> > > >>>> >>>     you can just delete it or replace it with a meaningful
>> > > >header
>> > > >>>> >>> line
>> > > >>>> >>>     with five fields and save the file under another name.
>> > > >>>> >>>
>> > > >>>> >>>     It looks as thought you have date-time as two fields. If
>> > > >so, you
>> > > >>>> >>> can
>> > > >>>> >>>     just read the first field if you only want the date:
>> > > >>>> >>>
>> > > >>>> >>>     # assume you have removed the first line
>> > > >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>> > > >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>> > > >>>> >>>
>> > > >>>> >>>     If you want the date/time:
>> > > >>>> >>>
>> > > >>>> >>>
>> > > >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>> > > >>>> >>> %H:%M:%S")
>> > > >>>> >>>
>> > > >>>> >>>     Jim
>> > > >>>> >>>
>> > > >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
>> > > >>>> >>> <[hidden email]> wrote:
>> > > >>>> >>>     > Dear all,
>> > > >>>> >>>     >
>> > > >>>> >>>     > I am dealing with the reading of a *.txt file.
>> > > >>>> >>>     > The txt file the following shape:
>> > > >>>> >>>     >
>> > > >>>> >>>     > 103001930 103001580 103001530
>> > > >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
>> > > >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>> > > >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>> > > >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
>> > > >>>> >>>     > 1998-10-01 04:00:00 0 0 0
>> > > >>>> >>>     > 1998-10-01 05:00:00 0 0 0
>> > > >>>> >>>     > 1998-10-01 06:00:00 0 0 0
>> > > >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
>> > > >>>> >>>     >
>> > > >>>> >>>     > If it is possible I have a coupe of questions, which
>> > > >>>> >>> will
>> > > >sound
>> > > >>>> >>> stupid but
>> > > >>>> >>>     > they are important to me in order to understand ho R
>> > > >>>> >>> deal
>> > > >with
>> > > >>>> >>> file
>> > > >>>> >>> or date.
>> > > >>>> >>>     >
>> > > >>>> >>>     > 1) Do I have to convert it to a *csv file?
>> > > >>>> >>>     > 2) Can a deal with space and not ","
>> > > >>>> >>>     > 3) How can I read date?
>> > > >>>> >>>     >
>> > > >>>> >>>     > thanks a lot to all of you,
>> > > >>>> >>>     > Thanks
>> > > >>>> >>>     >
>> > > >>>> >>>     >
>> > > >>>> >>>     > Diego
>> > > >>>> >>>     >
>> > > >>>> >>>     >         [[alternative HTML version deleted]]
>> > > >>>> >>>     >
>> > > >>>> >>>     > ______________________________________________
>> > > >>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE
>> and
>> > > >more,
>> > > >>>> >>> see
>> > > >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
>> > > >>>> >>>     > PLEASE do read the posting guide
>> > > >>>> >>> http://www.R-project.org/posting-guide.html
>> > > >>>> >>>     > and provide commented, minimal, self-contained,
>> > > >reproducible
>> > > >>>> >>> code.
>> > > >>>> >>>
>> > > >>>> >>>     ______________________________________________
>> > > >>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and
>> > > >more, see
>> > > >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
>> > > >>>> >>>     PLEASE do read the posting guide
>> > > >>>> >>> http://www.R-project.org/posting-guide.html
>> > > >>>> >>>     and provide commented, minimal, self-contained,
>> > > >reproducible
>> > > >>>> >>> code.
>> > > >>>> >>>
>> > > >>>> >>>
>> > > >>>> >>
>> > > >>>> >
>> > > >>>
>> > > >>>
>> > > >>
>> > > >
>> > > >______________________________________________
>> > > >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> > > >https://stat.ethz.ch/mailman/listinfo/r-help
>> > > >PLEASE do read the posting guide
>> > > >http://www.R-project.org/posting-guide.html
>> > > >and provide commented, minimal, self-contained, reproducible code.
>> > >
>> > > --
>> > > Sent from my phone. Please excuse my brevity.
>> > >
>> >
>> > [[alternative HTML version deleted]]
>> >
>> > ______________________________________________
>> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide http://www.R-project.org/posti
>> ng-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>> Osobní údaje: Informace o zpracování a ochraně osobních údajů obchodních
>> partnerů PRECHEZA a.s. jsou zveřejněny na: https://www.precheza.cz/zasady
>> -ochrany-osobnich-udaju/ | Information about processing and protection
>> of business partner’s personal data are available on website:
>> https://www.precheza.cz/en/personal-data-protection-principles/
>> Důvěrnost: Tento e-mail a jakékoliv k němu připojené dokumenty jsou
>> důvěrné a podléhají tomuto právně závaznému prohláąení o vyloučení
>> odpovědnosti: https://www.precheza.cz/01-dovetek/ | This email and any
>> documents attached to it may be confidential and are subject to the legally
>> binding disclaimer: https://www.precheza.cz/en/01-disclaimer/
>>
>>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

PIKAL Petr
In reply to this post by Diego Avesani
Hi

see in line

From: Diego Avesani <[hidden email]>
Sent: Wednesday, August 1, 2018 2:30 PM
To: PIKAL Petr <[hidden email]>
Cc: r-help mailing list <[hidden email]>
Subject: Re: [R] read txt file - date - no space

Dear Pikal, Deal all,

again really thank.

it seems not working.
Some specifications: My non data are -999, but I could change it.

You must change it to NA. How the poor R should know that -999 is missing and not a real value.

something like

Mydata[Mydata== -999] <- NA

should do it.

My final results is:


1        1  -55.86242 -55.84764660 -277.4775

2        2  -55.47554 -94.58921682 -277.4845

3        3  -55.47095 -99.20239198 -277.4709

4        4  -55.46470 -55.45952932 -392.9071

5        5  -55.43335 -55.40171682 -388.4110

6        6  -55.40108 -55.37399691 -332.9068

7        7  -55.39201 -55.35156250 -332.8902

8        8 -110.87184   0.16136188 -281.8230

9        9 -110.95077 -55.63856096 -332.9564

10      10 -157.64430  -0.06602705 -315.3840

11      11 -105.06157   0.11507675 -315.4152

12      12  -70.08677 -52.54501096 -316.7247

So it is not correct.
For example for the first day in my csv I would have expected 0.167.

I am going to post what you have suggested:

for  str(MyData)

Your data seems to be OK, so after you change your -999 to NA everything should be OK.

'data.frame':                 160008 obs. of  4 variables:
 $ date: POSIXct, format: "1998-01-10 00:00:00" "1998-01-10 01:00:00" "1998-01-10 02:00:00" "1998-01-10 03:00:00" ...
 $ str1: num  0.6 0.2 0.6 0 0 0 0 0.2 0.6 0.2 ...
 $ str2: num  0 0.2 0.2 0 0 0 0 0 0.2 0.4 ...
 $ str3: num  0 0.2 0.4 0.6 0 0 0 0 0 0.4 ...

unfortunately, I am not able to post

dput(str[1:20,])

it gives me

Error in str[1:20, ] : object of type 'closure' is not subsettable



dput(MyData[1:20,])



I would recommend you to spend some time reading R-intro which should be located in doc/manual folder of your installation. It could help you in many situations.



Cheers

Petr


Thanks again,
I hope that what I posted could be enough in order to help me.



Diego

On 1 August 2018 at 13:58, PIKAL Petr <[hidden email]<mailto:[hidden email]>> wrote:
Hi

I did not get through all answers you already got and you probably obtained similar advice as mine.

# read data (if you have csv file just use read.csv)
> test<-read.table("clipboard", header=T, sep=",")

# control your object(s)
> str(test)
'data.frame':   8 obs. of  4 variables:
 $ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
 $ str1: num  0.6 0.2 0.6 0 0 0 0 0.2
 $ str2: num  0 0.2 0.2 0 0 0 0 0
 $ str3: num  0 0.2 0.4 0.6 0 0 0 0

#if it is OK change first column to real date by POSIXct
test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")

#attach lubridate
> library(lubridate)

# aggregate your object(s) and use lubridate function

> aggregate(test[,-1], list(day(test$date)), mean)
  Group.1 str1 str2 str3
1      10  0.2 0.05 0.15

# or format function

> aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
     Group.1 str1 str2 str3
1 1998-01-10  0.2 0.05 0.15

If it does not work with your data you should post at least result of

str(yourdata)

or preferably

dput(yourdata[1:20,])

Cheers
Petr


> -----Original Message-----
> From: R-help <[hidden email]<mailto:[hidden email]>> On Behalf Of Diego Avesani
> Sent: Wednesday, August 1, 2018 8:55 AM
> To: Jeff Newmiller <[hidden email]<mailto:[hidden email]>>
> Cc: r-help mailing list <[hidden email]<mailto:[hidden email]>>
> Subject: Re: [R] read txt file - date - no space
>
> Dear all,
> I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat all again
> in order to understand.
> If I could I would like to start again, without mixing strategy and waiting for
> your advice.
>
> I am really appreciate you help, really really.
> Here my new file, a *.csv file (buy the way, it is possible to attach it in the
> mailing list?)
>
> date,str1,str2,str3
> 10/1/1998 0:00,0.6,0,0
> 10/1/1998 1:00,0.2,0.2,0.2
> 10/1/1998 2:00,0.6,0.2,0.4
> 10/1/1998 3:00,0,0,0.6
> 10/1/1998 4:00,0,0,0
> 10/1/1998 5:00,0,0,0
> 10/1/1998 6:00,0,0,0
> 10/1/1998 7:00,0.2,0,0
>
>
> I read it as:
> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>
> at this point I would like to have the daily mean.
> What would you suggest?
>
> Really Really thanks,
> You are my lifesaver
>
> Thanks
>
>
>
> Diego
>
>
> On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]<mailto:[hidden email]>> wrote:
>
> > ... and the most common source of NA values in time data is wrong
> > timezones. You really need to make sure the timezone that is assumed
> > when the character data are converted to POSIXt agrees with the data.
> > In most cases the easiest way to insure this is to use
> >
> > Sys.setenv(TZ="US/Pacific")
> >
> > or whatever timezone from
> >
> > OlsonNames()
> >
> > corresponds with your data. Execute this setenv function before the
> > strptime or as.POSIXct() function call.
> >
> > You can use
> >
> > MyData[ is.na<http://is.na>(MyData$datetime), ]
> >
> > to see which records are failing to convert time.
> >
> > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
> >
> > On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]<mailto:[hidden email]>>
> wrote:
> > >Hi Diego,
> > >I think the error is due to NA values in your data file. If I extend
> > >your example and run it, I get no errors:
> > >
> > >MyData<-read.table(text="103001930 103001580 103001530
> > >1998-10-01 00:00:00 0.6 0 0
> > >1998-10-01 01:00:00 0.2 0.2 0.2
> > >1998-10-01 02:00:00 0.6 0.2 0.4
> > >1998-10-01 03:00:00 0 0 0.6
> > >1998-10-01 04:00:00 0 0 0
> > >1998-10-01 05:00:00 0 0 0
> > >1998-10-01 06:00:00 0 0 0
> > >1998-10-01 07:00:00 0.2 0 0
> > >1998-10-01 08:00:00 0.6 0 0
> > >1998-10-01 09:00:00 0.2 0.2 0.2
> > >1998-10-01 10:00:00 0.6 0.2 0.4
> > >1998-10-01 11:00:00 0 0 0.6
> > >1998-10-01 12:00:00 0 0 0
> > >1998-10-01 13:00:00 0 0 0
> > >1998-10-01 14:00:00 0 0 0
> > >1998-10-01 15:00:00 0.2 0 0
> > >1998-10-01 16:00:00 0.6 0 0
> > >1998-10-01 17:00:00 0.2 0.2 0.2
> > >1998-10-01 18:00:00 0.6 0.2 0.4
> > >1998-10-01 19:00:00 0 0 0.6
> > >1998-10-01 20:00:00 0 0 0
> > >1998-10-01 21:00:00 0 0 0
> > >1998-10-01 22:00:00 0 0 0
> > >1998-10-01 23:00:00 0.2 0 0
> > >1998-10-02 00:00:00 0.6 0 0
> > >1998-10-02 01:00:00 0.2 0.2 0.2
> > >1998-10-02 02:00:00 0.6 0.2 0.4
> > >1998-10-02 03:00:00 0 0 0.6
> > >1998-10-02 04:00:00 0 0 0
> > >1998-10-02 05:00:00 0 0 0
> > >1998-10-02 06:00:00 0 0 0
> > >1998-10-02 07:00:00 0.2 0 0
> > >1998-10-02 08:00:00 0.6 0 0
> > >1998-10-02 09:00:00 0.2 0.2 0.2
> > >1998-10-02 10:00:00 0.6 0.2 0.4
> > >1998-10-02 11:00:00 0 0 0.6
> > >1998-10-02 12:00:00 0 0 0
> > >1998-10-02 13:00:00 0 0 0
> > >1998-10-02 14:00:00 0 0 0
> > >1998-10-02 15:00:00 0.2 0 0
> > >1998-10-02 16:00:00 0.6 0 0
> > >1998-10-02 17:00:00 0.2 0.2 0.2
> > >1998-10-02 18:00:00 0.6 0.2 0.4
> > >1998-10-02 19:00:00 0 0 0.6
> > >1998-10-02 20:00:00 0 0 0
> > >1998-10-02 21:00:00 0 0 0
> > >1998-10-02 22:00:00 0 0 0
> > >1998-10-02 23:00:00 0.2 0 0",
> > >skip=1,stringsAsFactors=FALSE)
> > >names(MyData)<-c("date","time","st1","st2","st3")
> > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > > format="%Y-%m-%d %H:%M:%S")
> > >MyData$datetime
> > >st1_daily<-by(MyData$st1,MyData$date,mean)
> > >st2_daily<-by(MyData$st2,MyData$date,mean)
> > >st3_daily<-by(MyData$st3,MyData$date,mean)
> > >st1_daily
> > >st2_daily
> > >st3_daily
> > >
> > >Try adding na.rm=TRUE to the "by" calls:
> > >
> > >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
> > >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
> > >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
> > >
> > >Jim
> > >
> > >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
> > ><[hidden email]<mailto:[hidden email]>> wrote:
> > >> Dear all,
> > >>
> > >> I have still problem with date.
> > >> Could you please tel me how to use POSIXct.
> > >> Indeed I have found this command:
> > >> timeAverage, but I am not able to convert MyDate to properly date.
> > >>
> > >> Thank a lot
> > >> I hope to no bother you, at least too much
> > >>
> > >>
> > >> Diego
> > >>
> > >>
> > >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]<mailto:[hidden email]>>
> > >wrote:
> > >>>
> > >>> Dear Jim, Dear all,
> > >>>
> > >>> thanks a lot.
> > >>>
> > >>> Unfortunately, I get the following error:
> > >>>
> > >>>
> > >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
> > >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
> > >925L,  :
> > >>>   arguments must have same length
> > >>>
> > >>>
> > >>> This is particularly strange. indeed, if I apply
> > >>>
> > >>>
> > >>> mean(MyData$str1,na.rm=TRUE)
> > >>>
> > >>>
> > >>> it works
> > >>>
> > >>>
> > >>> Sorry, I have to learn a lot.
> > >>> You are really boosting me
> > >>>
> > >>> Diego
> > >>>
> > >>>
> > >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]<mailto:[hidden email]>> wrote:
> > >>>>
> > >>>> Hi Diego,
> > >>>> One way you can get daily means is:
> > >>>>
> > >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
> > >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
> > >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
> > >>>>
> > >>>> Jim
> > >>>>
> > >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
> > ><[hidden email]<mailto:[hidden email]>>
> > >>>> wrote:
> > >>>> > Dear all,
> > >>>> > I have found the error, my fault. Sorry.
> > >>>> > There was an extra come in the headers line.
> > >>>> > Thanks again.
> > >>>> >
> > >>>> > If I can I would like to ask you another questions about the
> > >imported
> > >>>> > data.
> > >>>> > I would like to compute the daily average of the different date.
> > >>>> > Basically I
> > >>>> > have hourly data, I would like to ave the daily mean of them.
> > >>>> >
> > >>>> > Is there some special commands?
> > >>>> >
> > >>>> > Thanks a lot.
> > >>>> >
> > >>>> >
> > >>>> > Diego
> > >>>> >
> > >>>> >
> > >>>> > On 31 July 2018 at 10:40, Diego Avesani
> > >>>> > <[hidden email]<mailto:[hidden email]>>
> > >>>> > wrote:
> > >>>> >>
> > >>>> >> Dear all,
> > >>>> >> I move to csv file because originally the date where in csv
> > >file.
> > >>>> >> In addition, due to the fact that, as you told me, read.csv is
> > >>>> >> a special case of read.table, I prefer start to learn from the
> > >>>> >> simplest
> > >one.
> > >>>> >> After that, I will try also the *.txt format.
> > >>>> >>
> > >>>> >> with read.csv, something strange happened:
> > >>>> >>
> > >>>> >> This us now the file:
> > >>>> >>
> > >>>> >> date,st1,st2,st3,
> > >>>> >> 10/1/1998 0:00,0.6,0,0
> > >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
> > >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
> > >>>> >> 10/1/1998 3:00,0,0,0.6
> > >>>> >> 10/1/1998 4:00,0,0,0
> > >>>> >> 10/1/1998 5:00,0,0,0
> > >>>> >> 10/1/1998 6:00,0,0,0
> > >>>> >> 10/1/1998 7:00,0.2,0,0
> > >>>> >> 10/1/1998 8:00,0.6,0.2,0
> > >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
> > >>>> >> 10/1/1998 10:00,0,0.4,0.2
> > >>>> >>
> > >>>> >> When I apply:
> > >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> > >>>> >>
> > >>>> >> this is the results:
> > >>>> >>
> > >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
> > >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
> > >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
> > >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
> > >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
> > >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
> > >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
> > >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
> > >>>> >>
> > >>>> >> I do not understand why.
> > >>>> >> Something wrong with date?
> > >>>> >>
> > >>>> >> really really thanks,
> > >>>> >> I appreciate a lot all your helps.
> > >>>> >>
> > >>>> >> Diedro
> > >>>> >>
> > >>>> >>
> > >>>> >> Diego
> > >>>> >>
> > >>>> >>
> > >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]<mailto:[hidden email]>>
> > >wrote:
> > >>>> >>>
> > >>>> >>> Or, without removing the first line
> > >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE,
> > >>>> >>> skip=1)
> > >>>> >>>
> > >>>> >>> Another alternative,
> > >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> > >>>> >>> since the dates appear to be in the default format.
> > >>>> >>> (I generally prefer to work with datetimes in POSIXct class
> > >rather
> > >>>> >>> than
> > >>>> >>> POSIXlt class)
> > >>>> >>>
> > >>>> >>> -Don
> > >>>> >>>
> > >>>> >>> --
> > >>>> >>> Don MacQueen
> > >>>> >>> Lawrence Livermore National Laboratory
> > >>>> >>> 7000 East Ave., L-627
> > >>>> >>> Livermore, CA 94550
> > >>>> >>> 925-423-1062
> > >>>> >>> Lab cell 925-724-7509
> > >>>> >>>
> > >>>> >>>
> > >>>> >>>
> > >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
> > >>>> >>> <[hidden email]<mailto:[hidden email]> on behalf of
> > >[hidden email]<mailto:[hidden email]>>
> > >>>> >>> wrote:
> > >>>> >>>
> > >>>> >>>     Hi Diego,
> > >>>> >>>     You may have to do some conversion as you have three
> > >>>> >>> fields
> > >in
> > >>>> >>> the
> > >>>> >>>     first line using the default space separator and five
> > >fields in
> > >>>> >>>     subsequent lines. If the first line doesn't contain any
> > >important
> > >>>> >>> data
> > >>>> >>>     you can just delete it or replace it with a meaningful
> > >header
> > >>>> >>> line
> > >>>> >>>     with five fields and save the file under another name.
> > >>>> >>>
> > >>>> >>>     It looks as thought you have date-time as two fields. If
> > >so, you
> > >>>> >>> can
> > >>>> >>>     just read the first field if you only want the date:
> > >>>> >>>
> > >>>> >>>     # assume you have removed the first line
> > >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
> > >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
> > >>>> >>>
> > >>>> >>>     If you want the date/time:
> > >>>> >>>
> > >>>> >>>
> > >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> > >>>> >>> %H:%M:%S")
> > >>>> >>>
> > >>>> >>>     Jim
> > >>>> >>>
> > >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
> > >>>> >>> <[hidden email]<mailto:[hidden email]>> wrote:
> > >>>> >>>     > Dear all,
> > >>>> >>>     >
> > >>>> >>>     > I am dealing with the reading of a *.txt file.
> > >>>> >>>     > The txt file the following shape:
> > >>>> >>>     >
> > >>>> >>>     > 103001930 103001580 103001530
> > >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
> > >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
> > >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
> > >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
> > >>>> >>>     > 1998-10-01 04:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 05:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 06:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
> > >>>> >>>     >
> > >>>> >>>     > If it is possible I have a coupe of questions, which
> > >>>> >>> will
> > >sound
> > >>>> >>> stupid but
> > >>>> >>>     > they are important to me in order to understand ho R
> > >>>> >>> deal
> > >with
> > >>>> >>> file
> > >>>> >>> or date.
> > >>>> >>>     >
> > >>>> >>>     > 1) Do I have to convert it to a *csv file?
> > >>>> >>>     > 2) Can a deal with space and not ","
> > >>>> >>>     > 3) How can I read date?
> > >>>> >>>     >
> > >>>> >>>     > thanks a lot to all of you,
> > >>>> >>>     > Thanks
> > >>>> >>>     >
> > >>>> >>>     >
> > >>>> >>>     > Diego
> > >>>> >>>     >
> > >>>> >>>     >         [[alternative HTML version deleted]]
> > >>>> >>>     >
> > >>>> >>>     > ______________________________________________
> > >>>> >>>     > [hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and
> > >more,
> > >>>> >>> see
> > >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
> > >>>> >>>     > PLEASE do read the posting guide
> > >>>> >>> http://www.R-project.org/posting-guide.html
> > >>>> >>>     > and provide commented, minimal, self-contained,
> > >reproducible
> > >>>> >>> code.
> > >>>> >>>
> > >>>> >>>     ______________________________________________
> > >>>> >>>     [hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and
> > >more, see
> > >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
> > >>>> >>>     PLEASE do read the posting guide
> > >>>> >>> http://www.R-project.org/posting-guide.html
> > >>>> >>>     and provide commented, minimal, self-contained,
> > >reproducible
> > >>>> >>> code.
> > >>>> >>>
> > >>>> >>>
> > >>>> >>
> > >>>> >
> > >>>
> > >>>
> > >>
> > >
> > >______________________________________________
> > >[hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and more, see
> > >https://stat.ethz.ch/mailman/listinfo/r-help
> > >PLEASE do read the posting guide
> > >http://www.R-project.org/posting-guide.html
> > >and provide commented, minimal, self-contained, reproducible code.
> >
> > --
> > Sent from my phone. Please excuse my brevity.
> >
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
Osobní údaje: Informace o zpracování a ochraně osobních údajů obchodních partnerů PRECHEZA a.s. jsou zveřejněny na: https://www.precheza.cz/zasady-ochrany-osobnich-udaju/ | Information about processing and protection of business partner’s personal data are available on website: https://www.precheza.cz/en/personal-data-protection-principles/
Důvěrnost: Tento e-mail a jakékoliv k němu připojené dokumenty jsou důvěrné a podléhají tomuto právně závaznému prohláąení o vyloučení odpovědnosti: https://www.precheza.cz/01-dovetek/ | This email and any documents attached to it may be confidential and are subject to the legally binding disclaimer: https://www.precheza.cz/en/01-disclaimer/


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

jholtman
In reply to this post by Diego Avesani
Try this:

> library(lubridate)
> library(tidyverse)
> input <- read.csv(text = "date,str1,str2,str3
+ 10/1/1998 0:00,0.6,0,0
+                   10/1/1998 1:00,0.2,0.2,0.2
+                   10/1/1998 2:00,0.6,0.2,0.4
+                   10/1/1998 3:00,0,0,0.6
+                   10/1/1998 4:00,0,0,0
+                   10/1/1998 5:00,0,0,0
+                   10/1/1998 6:00,0,0,0
+                   10/1/1998 7:00,0.2,0,0", as.is = TRUE)
> # convert the date and add the "day" so summarize
> input <- input %>%
+   mutate(date = mdy_hm(date),
+          day = floor_date(date, unit = 'day')
+   )
>
> by_day <- input %>%
+   group_by(day) %>%
+   summarise(m_s1 = mean(str1),
+             m_s2 = mean(str2),
+             m_s3 = mean(str3)
+   )
>
> by_day
# A tibble: 1 x 4
  day                  m_s1   m_s2  m_s3
  <dttm>              <dbl>  <dbl> <dbl>
1 1998-10-01 00:00:00 0.200 0.0500 0.150

Jim Holtman
*Data Munger Guru*


*What is the problem that you are trying to solve?Tell me what you want to
do, not how you want to do it.*


On Tue, Jul 31, 2018 at 11:54 PM Diego Avesani <[hidden email]>
wrote:

> Dear all,
> I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat
> all again in order to understand.
> If I could I would like to start again, without mixing strategy and waiting
> for your advice.
>
> I am really appreciate you help, really really.
> Here my new file, a *.csv file (buy the way, it is possible to attach it in
> the mailing list?)
>
> date,str1,str2,str3
> 10/1/1998 0:00,0.6,0,0
> 10/1/1998 1:00,0.2,0.2,0.2
> 10/1/1998 2:00,0.6,0.2,0.4
> 10/1/1998 3:00,0,0,0.6
> 10/1/1998 4:00,0,0,0
> 10/1/1998 5:00,0,0,0
> 10/1/1998 6:00,0,0,0
> 10/1/1998 7:00,0.2,0,0
>
>
> I read it as:
> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>
> at this point I would like to have the daily mean.
> What would you suggest?
>
> Really Really thanks,
> You are my lifesaver
>
> Thanks
>
>
>
> Diego
>
>
> On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]>
> wrote:
>
> > ... and the most common source of NA values in time data is wrong
> > timezones. You really need to make sure the timezone that is assumed when
> > the character data are converted to POSIXt agrees with the data. In most
> > cases the easiest way to insure this is to use
> >
> > Sys.setenv(TZ="US/Pacific")
> >
> > or whatever timezone from
> >
> > OlsonNames()
> >
> > corresponds with your data. Execute this setenv function before the
> > strptime or as.POSIXct() function call.
> >
> > You can use
> >
> > MyData[ is.na(MyData$datetime), ]
> >
> > to see which records are failing to convert time.
> >
> > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
> >
> > On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]> wrote:
> > >Hi Diego,
> > >I think the error is due to NA values in your data file. If I extend
> > >your example and run it, I get no errors:
> > >
> > >MyData<-read.table(text="103001930 103001580 103001530
> > >1998-10-01 00:00:00 0.6 0 0
> > >1998-10-01 01:00:00 0.2 0.2 0.2
> > >1998-10-01 02:00:00 0.6 0.2 0.4
> > >1998-10-01 03:00:00 0 0 0.6
> > >1998-10-01 04:00:00 0 0 0
> > >1998-10-01 05:00:00 0 0 0
> > >1998-10-01 06:00:00 0 0 0
> > >1998-10-01 07:00:00 0.2 0 0
> > >1998-10-01 08:00:00 0.6 0 0
> > >1998-10-01 09:00:00 0.2 0.2 0.2
> > >1998-10-01 10:00:00 0.6 0.2 0.4
> > >1998-10-01 11:00:00 0 0 0.6
> > >1998-10-01 12:00:00 0 0 0
> > >1998-10-01 13:00:00 0 0 0
> > >1998-10-01 14:00:00 0 0 0
> > >1998-10-01 15:00:00 0.2 0 0
> > >1998-10-01 16:00:00 0.6 0 0
> > >1998-10-01 17:00:00 0.2 0.2 0.2
> > >1998-10-01 18:00:00 0.6 0.2 0.4
> > >1998-10-01 19:00:00 0 0 0.6
> > >1998-10-01 20:00:00 0 0 0
> > >1998-10-01 21:00:00 0 0 0
> > >1998-10-01 22:00:00 0 0 0
> > >1998-10-01 23:00:00 0.2 0 0
> > >1998-10-02 00:00:00 0.6 0 0
> > >1998-10-02 01:00:00 0.2 0.2 0.2
> > >1998-10-02 02:00:00 0.6 0.2 0.4
> > >1998-10-02 03:00:00 0 0 0.6
> > >1998-10-02 04:00:00 0 0 0
> > >1998-10-02 05:00:00 0 0 0
> > >1998-10-02 06:00:00 0 0 0
> > >1998-10-02 07:00:00 0.2 0 0
> > >1998-10-02 08:00:00 0.6 0 0
> > >1998-10-02 09:00:00 0.2 0.2 0.2
> > >1998-10-02 10:00:00 0.6 0.2 0.4
> > >1998-10-02 11:00:00 0 0 0.6
> > >1998-10-02 12:00:00 0 0 0
> > >1998-10-02 13:00:00 0 0 0
> > >1998-10-02 14:00:00 0 0 0
> > >1998-10-02 15:00:00 0.2 0 0
> > >1998-10-02 16:00:00 0.6 0 0
> > >1998-10-02 17:00:00 0.2 0.2 0.2
> > >1998-10-02 18:00:00 0.6 0.2 0.4
> > >1998-10-02 19:00:00 0 0 0.6
> > >1998-10-02 20:00:00 0 0 0
> > >1998-10-02 21:00:00 0 0 0
> > >1998-10-02 22:00:00 0 0 0
> > >1998-10-02 23:00:00 0.2 0 0",
> > >skip=1,stringsAsFactors=FALSE)
> > >names(MyData)<-c("date","time","st1","st2","st3")
> > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > > format="%Y-%m-%d %H:%M:%S")
> > >MyData$datetime
> > >st1_daily<-by(MyData$st1,MyData$date,mean)
> > >st2_daily<-by(MyData$st2,MyData$date,mean)
> > >st3_daily<-by(MyData$st3,MyData$date,mean)
> > >st1_daily
> > >st2_daily
> > >st3_daily
> > >
> > >Try adding na.rm=TRUE to the "by" calls:
> > >
> > >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
> > >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
> > >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
> > >
> > >Jim
> > >
> > >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
> > ><[hidden email]> wrote:
> > >> Dear all,
> > >>
> > >> I have still problem with date.
> > >> Could you please tel me how to use POSIXct.
> > >> Indeed I have found this command:
> > >> timeAverage, but I am not able to convert MyDate to properly date.
> > >>
> > >> Thank a lot
> > >> I hope to no bother you, at least too much
> > >>
> > >>
> > >> Diego
> > >>
> > >>
> > >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]>
> > >wrote:
> > >>>
> > >>> Dear Jim, Dear all,
> > >>>
> > >>> thanks a lot.
> > >>>
> > >>> Unfortunately, I get the following error:
> > >>>
> > >>>
> > >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
> > >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
> > >925L,  :
> > >>>   arguments must have same length
> > >>>
> > >>>
> > >>> This is particularly strange. indeed, if I apply
> > >>>
> > >>>
> > >>> mean(MyData$str1,na.rm=TRUE)
> > >>>
> > >>>
> > >>> it works
> > >>>
> > >>>
> > >>> Sorry, I have to learn a lot.
> > >>> You are really boosting me
> > >>>
> > >>> Diego
> > >>>
> > >>>
> > >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
> > >>>>
> > >>>> Hi Diego,
> > >>>> One way you can get daily means is:
> > >>>>
> > >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
> > >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
> > >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
> > >>>>
> > >>>> Jim
> > >>>>
> > >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
> > ><[hidden email]>
> > >>>> wrote:
> > >>>> > Dear all,
> > >>>> > I have found the error, my fault. Sorry.
> > >>>> > There was an extra come in the headers line.
> > >>>> > Thanks again.
> > >>>> >
> > >>>> > If I can I would like to ask you another questions about the
> > >imported
> > >>>> > data.
> > >>>> > I would like to compute the daily average of the different date.
> > >>>> > Basically I
> > >>>> > have hourly data, I would like to ave the daily mean of them.
> > >>>> >
> > >>>> > Is there some special commands?
> > >>>> >
> > >>>> > Thanks a lot.
> > >>>> >
> > >>>> >
> > >>>> > Diego
> > >>>> >
> > >>>> >
> > >>>> > On 31 July 2018 at 10:40, Diego Avesani <[hidden email]>
> > >>>> > wrote:
> > >>>> >>
> > >>>> >> Dear all,
> > >>>> >> I move to csv file because originally the date where in csv
> > >file.
> > >>>> >> In addition, due to the fact that, as you told me, read.csv is a
> > >>>> >> special
> > >>>> >> case of read.table, I prefer start to learn from the simplest
> > >one.
> > >>>> >> After that, I will try also the *.txt format.
> > >>>> >>
> > >>>> >> with read.csv, something strange happened:
> > >>>> >>
> > >>>> >> This us now the file:
> > >>>> >>
> > >>>> >> date,st1,st2,st3,
> > >>>> >> 10/1/1998 0:00,0.6,0,0
> > >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
> > >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
> > >>>> >> 10/1/1998 3:00,0,0,0.6
> > >>>> >> 10/1/1998 4:00,0,0,0
> > >>>> >> 10/1/1998 5:00,0,0,0
> > >>>> >> 10/1/1998 6:00,0,0,0
> > >>>> >> 10/1/1998 7:00,0.2,0,0
> > >>>> >> 10/1/1998 8:00,0.6,0.2,0
> > >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
> > >>>> >> 10/1/1998 10:00,0,0.4,0.2
> > >>>> >>
> > >>>> >> When I apply:
> > >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> > >>>> >>
> > >>>> >> this is the results:
> > >>>> >>
> > >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
> > >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
> > >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
> > >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
> > >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
> > >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
> > >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
> > >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
> > >>>> >>
> > >>>> >> I do not understand why.
> > >>>> >> Something wrong with date?
> > >>>> >>
> > >>>> >> really really thanks,
> > >>>> >> I appreciate a lot all your helps.
> > >>>> >>
> > >>>> >> Diedro
> > >>>> >>
> > >>>> >>
> > >>>> >> Diego
> > >>>> >>
> > >>>> >>
> > >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]>
> > >wrote:
> > >>>> >>>
> > >>>> >>> Or, without removing the first line
> > >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
> > >>>> >>>
> > >>>> >>> Another alternative,
> > >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> > >>>> >>> since the dates appear to be in the default format.
> > >>>> >>> (I generally prefer to work with datetimes in POSIXct class
> > >rather
> > >>>> >>> than
> > >>>> >>> POSIXlt class)
> > >>>> >>>
> > >>>> >>> -Don
> > >>>> >>>
> > >>>> >>> --
> > >>>> >>> Don MacQueen
> > >>>> >>> Lawrence Livermore National Laboratory
> > >>>> >>> 7000 East Ave., L-627
> > >>>> >>> Livermore, CA 94550
> > >>>> >>> 925-423-1062
> > >>>> >>> Lab cell 925-724-7509
> > >>>> >>>
> > >>>> >>>
> > >>>> >>>
> > >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
> > >>>> >>> <[hidden email] on behalf of
> > >[hidden email]>
> > >>>> >>> wrote:
> > >>>> >>>
> > >>>> >>>     Hi Diego,
> > >>>> >>>     You may have to do some conversion as you have three fields
> > >in
> > >>>> >>> the
> > >>>> >>>     first line using the default space separator and five
> > >fields in
> > >>>> >>>     subsequent lines. If the first line doesn't contain any
> > >important
> > >>>> >>> data
> > >>>> >>>     you can just delete it or replace it with a meaningful
> > >header
> > >>>> >>> line
> > >>>> >>>     with five fields and save the file under another name.
> > >>>> >>>
> > >>>> >>>     It looks as thought you have date-time as two fields. If
> > >so, you
> > >>>> >>> can
> > >>>> >>>     just read the first field if you only want the date:
> > >>>> >>>
> > >>>> >>>     # assume you have removed the first line
> > >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
> > >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
> > >>>> >>>
> > >>>> >>>     If you want the date/time:
> > >>>> >>>
> > >>>> >>>
> > >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> > >>>> >>> %H:%M:%S")
> > >>>> >>>
> > >>>> >>>     Jim
> > >>>> >>>
> > >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
> > >>>> >>> <[hidden email]> wrote:
> > >>>> >>>     > Dear all,
> > >>>> >>>     >
> > >>>> >>>     > I am dealing with the reading of a *.txt file.
> > >>>> >>>     > The txt file the following shape:
> > >>>> >>>     >
> > >>>> >>>     > 103001930 103001580 103001530
> > >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
> > >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
> > >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
> > >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
> > >>>> >>>     > 1998-10-01 04:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 05:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 06:00:00 0 0 0
> > >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
> > >>>> >>>     >
> > >>>> >>>     > If it is possible I have a coupe of questions, which will
> > >sound
> > >>>> >>> stupid but
> > >>>> >>>     > they are important to me in order to understand ho R deal
> > >with
> > >>>> >>> file
> > >>>> >>> or date.
> > >>>> >>>     >
> > >>>> >>>     > 1) Do I have to convert it to a *csv file?
> > >>>> >>>     > 2) Can a deal with space and not ","
> > >>>> >>>     > 3) How can I read date?
> > >>>> >>>     >
> > >>>> >>>     > thanks a lot to all of you,
> > >>>> >>>     > Thanks
> > >>>> >>>     >
> > >>>> >>>     >
> > >>>> >>>     > Diego
> > >>>> >>>     >
> > >>>> >>>     >         [[alternative HTML version deleted]]
> > >>>> >>>     >
> > >>>> >>>     > ______________________________________________
> > >>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and
> > >more,
> > >>>> >>> see
> > >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
> > >>>> >>>     > PLEASE do read the posting guide
> > >>>> >>> http://www.R-project.org/posting-guide.html
> > >>>> >>>     > and provide commented, minimal, self-contained,
> > >reproducible
> > >>>> >>> code.
> > >>>> >>>
> > >>>> >>>     ______________________________________________
> > >>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and
> > >more, see
> > >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
> > >>>> >>>     PLEASE do read the posting guide
> > >>>> >>> http://www.R-project.org/posting-guide.html
> > >>>> >>>     and provide commented, minimal, self-contained,
> > >reproducible
> > >>>> >>> code.
> > >>>> >>>
> > >>>> >>>
> > >>>> >>
> > >>>> >
> > >>>
> > >>>
> > >>
> > >
> > >______________________________________________
> > >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > >https://stat.ethz.ch/mailman/listinfo/r-help
> > >PLEASE do read the posting guide
> > >http://www.R-project.org/posting-guide.html
> > >and provide commented, minimal, self-contained, reproducible code.
> >
> > --
> > Sent from my phone. Please excuse my brevity.
> >
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear

I have check the one of the line that gives me problem. I mean, which give
NA after R processing. I think that is similar to the others:

10/12/1998 10:00,0,0,0
10/12/1998 11:00,0,0,0
10/12/1998 12:00,0,0,0
10/12/1998 13:00,0,0,0
10/12/1998 14:00,0,0,0
10/12/1998 15:00,0,0,0
10/12/1998 16:00,0,0,0
10/12/1998 17:00,0,0,0

@jim: It seems that you suggestion is focus on reading data from the
terminal. It is possible to apply it to a *.csv file?

@Pikal: Could it be that there are some date conversion error?

Thanks again,
Diego


Diego


On 1 August 2018 at 17:01, jim holtman <[hidden email]> wrote:

>
> Try this:
>
> > library(lubridate)
> > library(tidyverse)
> > input <- read.csv(text = "date,str1,str2,str3
> + 10/1/1998 0:00,0.6,0,0
> +                   10/1/1998 1:00,0.2,0.2,0.2
> +                   10/1/1998 2:00,0.6,0.2,0.4
> +                   10/1/1998 3:00,0,0,0.6
> +                   10/1/1998 4:00,0,0,0
> +                   10/1/1998 5:00,0,0,0
> +                   10/1/1998 6:00,0,0,0
> +                   10/1/1998 7:00,0.2,0,0", as.is = TRUE)
> > # convert the date and add the "day" so summarize
> > input <- input %>%
> +   mutate(date = mdy_hm(date),
> +          day = floor_date(date, unit = 'day')
> +   )
> >
> > by_day <- input %>%
> +   group_by(day) %>%
> +   summarise(m_s1 = mean(str1),
> +             m_s2 = mean(str2),
> +             m_s3 = mean(str3)
> +   )
> >
> > by_day
> # A tibble: 1 x 4
>   day                  m_s1   m_s2  m_s3
>   <dttm>              <dbl>  <dbl> <dbl>
> 1 1998-10-01 00:00:00 0.200 0.0500 0.150
>
> Jim Holtman
> *Data Munger Guru*
>
>
> *What is the problem that you are trying to solve?Tell me what you want to
> do, not how you want to do it.*
>
>
> On Tue, Jul 31, 2018 at 11:54 PM Diego Avesani <[hidden email]>
> wrote:
>
>> Dear all,
>> I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat
>> all again in order to understand.
>> If I could I would like to start again, without mixing strategy and
>> waiting
>> for your advice.
>>
>> I am really appreciate you help, really really.
>> Here my new file, a *.csv file (buy the way, it is possible to attach it
>> in
>> the mailing list?)
>>
>> date,str1,str2,str3
>> 10/1/1998 0:00,0.6,0,0
>> 10/1/1998 1:00,0.2,0.2,0.2
>> 10/1/1998 2:00,0.6,0.2,0.4
>> 10/1/1998 3:00,0,0,0.6
>> 10/1/1998 4:00,0,0,0
>> 10/1/1998 5:00,0,0,0
>> 10/1/1998 6:00,0,0,0
>> 10/1/1998 7:00,0.2,0,0
>>
>>
>> I read it as:
>> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>>
>> at this point I would like to have the daily mean.
>> What would you suggest?
>>
>> Really Really thanks,
>> You are my lifesaver
>>
>> Thanks
>>
>>
>>
>> Diego
>>
>>
>> On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]>
>> wrote:
>>
>> > ... and the most common source of NA values in time data is wrong
>> > timezones. You really need to make sure the timezone that is assumed
>> when
>> > the character data are converted to POSIXt agrees with the data. In most
>> > cases the easiest way to insure this is to use
>> >
>> > Sys.setenv(TZ="US/Pacific")
>> >
>> > or whatever timezone from
>> >
>> > OlsonNames()
>> >
>> > corresponds with your data. Execute this setenv function before the
>> > strptime or as.POSIXct() function call.
>> >
>> > You can use
>> >
>> > MyData[ is.na(MyData$datetime), ]
>> >
>> > to see which records are failing to convert time.
>> >
>> > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
>> >
>> > On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]>
>> wrote:
>> > >Hi Diego,
>> > >I think the error is due to NA values in your data file. If I extend
>> > >your example and run it, I get no errors:
>> > >
>> > >MyData<-read.table(text="103001930 103001580 103001530
>> > >1998-10-01 00:00:00 0.6 0 0
>> > >1998-10-01 01:00:00 0.2 0.2 0.2
>> > >1998-10-01 02:00:00 0.6 0.2 0.4
>> > >1998-10-01 03:00:00 0 0 0.6
>> > >1998-10-01 04:00:00 0 0 0
>> > >1998-10-01 05:00:00 0 0 0
>> > >1998-10-01 06:00:00 0 0 0
>> > >1998-10-01 07:00:00 0.2 0 0
>> > >1998-10-01 08:00:00 0.6 0 0
>> > >1998-10-01 09:00:00 0.2 0.2 0.2
>> > >1998-10-01 10:00:00 0.6 0.2 0.4
>> > >1998-10-01 11:00:00 0 0 0.6
>> > >1998-10-01 12:00:00 0 0 0
>> > >1998-10-01 13:00:00 0 0 0
>> > >1998-10-01 14:00:00 0 0 0
>> > >1998-10-01 15:00:00 0.2 0 0
>> > >1998-10-01 16:00:00 0.6 0 0
>> > >1998-10-01 17:00:00 0.2 0.2 0.2
>> > >1998-10-01 18:00:00 0.6 0.2 0.4
>> > >1998-10-01 19:00:00 0 0 0.6
>> > >1998-10-01 20:00:00 0 0 0
>> > >1998-10-01 21:00:00 0 0 0
>> > >1998-10-01 22:00:00 0 0 0
>> > >1998-10-01 23:00:00 0.2 0 0
>> > >1998-10-02 00:00:00 0.6 0 0
>> > >1998-10-02 01:00:00 0.2 0.2 0.2
>> > >1998-10-02 02:00:00 0.6 0.2 0.4
>> > >1998-10-02 03:00:00 0 0 0.6
>> > >1998-10-02 04:00:00 0 0 0
>> > >1998-10-02 05:00:00 0 0 0
>> > >1998-10-02 06:00:00 0 0 0
>> > >1998-10-02 07:00:00 0.2 0 0
>> > >1998-10-02 08:00:00 0.6 0 0
>> > >1998-10-02 09:00:00 0.2 0.2 0.2
>> > >1998-10-02 10:00:00 0.6 0.2 0.4
>> > >1998-10-02 11:00:00 0 0 0.6
>> > >1998-10-02 12:00:00 0 0 0
>> > >1998-10-02 13:00:00 0 0 0
>> > >1998-10-02 14:00:00 0 0 0
>> > >1998-10-02 15:00:00 0.2 0 0
>> > >1998-10-02 16:00:00 0.6 0 0
>> > >1998-10-02 17:00:00 0.2 0.2 0.2
>> > >1998-10-02 18:00:00 0.6 0.2 0.4
>> > >1998-10-02 19:00:00 0 0 0.6
>> > >1998-10-02 20:00:00 0 0 0
>> > >1998-10-02 21:00:00 0 0 0
>> > >1998-10-02 22:00:00 0 0 0
>> > >1998-10-02 23:00:00 0.2 0 0",
>> > >skip=1,stringsAsFactors=FALSE)
>> > >names(MyData)<-c("date","time","st1","st2","st3")
>> > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
>> > > format="%Y-%m-%d %H:%M:%S")
>> > >MyData$datetime
>> > >st1_daily<-by(MyData$st1,MyData$date,mean)
>> > >st2_daily<-by(MyData$st2,MyData$date,mean)
>> > >st3_daily<-by(MyData$st3,MyData$date,mean)
>> > >st1_daily
>> > >st2_daily
>> > >st3_daily
>> > >
>> > >Try adding na.rm=TRUE to the "by" calls:
>> > >
>> > >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
>> > >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
>> > >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
>> > >
>> > >Jim
>> > >
>> > >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
>> > ><[hidden email]> wrote:
>> > >> Dear all,
>> > >>
>> > >> I have still problem with date.
>> > >> Could you please tel me how to use POSIXct.
>> > >> Indeed I have found this command:
>> > >> timeAverage, but I am not able to convert MyDate to properly date.
>> > >>
>> > >> Thank a lot
>> > >> I hope to no bother you, at least too much
>> > >>
>> > >>
>> > >> Diego
>> > >>
>> > >>
>> > >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]>
>> > >wrote:
>> > >>>
>> > >>> Dear Jim, Dear all,
>> > >>>
>> > >>> thanks a lot.
>> > >>>
>> > >>> Unfortunately, I get the following error:
>> > >>>
>> > >>>
>> > >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
>> > >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
>> > >925L,  :
>> > >>>   arguments must have same length
>> > >>>
>> > >>>
>> > >>> This is particularly strange. indeed, if I apply
>> > >>>
>> > >>>
>> > >>> mean(MyData$str1,na.rm=TRUE)
>> > >>>
>> > >>>
>> > >>> it works
>> > >>>
>> > >>>
>> > >>> Sorry, I have to learn a lot.
>> > >>> You are really boosting me
>> > >>>
>> > >>> Diego
>> > >>>
>> > >>>
>> > >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
>> > >>>>
>> > >>>> Hi Diego,
>> > >>>> One way you can get daily means is:
>> > >>>>
>> > >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
>> > >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
>> > >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
>> > >>>>
>> > >>>> Jim
>> > >>>>
>> > >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
>> > ><[hidden email]>
>> > >>>> wrote:
>> > >>>> > Dear all,
>> > >>>> > I have found the error, my fault. Sorry.
>> > >>>> > There was an extra come in the headers line.
>> > >>>> > Thanks again.
>> > >>>> >
>> > >>>> > If I can I would like to ask you another questions about the
>> > >imported
>> > >>>> > data.
>> > >>>> > I would like to compute the daily average of the different date.
>> > >>>> > Basically I
>> > >>>> > have hourly data, I would like to ave the daily mean of them.
>> > >>>> >
>> > >>>> > Is there some special commands?
>> > >>>> >
>> > >>>> > Thanks a lot.
>> > >>>> >
>> > >>>> >
>> > >>>> > Diego
>> > >>>> >
>> > >>>> >
>> > >>>> > On 31 July 2018 at 10:40, Diego Avesani <[hidden email]
>> >
>> > >>>> > wrote:
>> > >>>> >>
>> > >>>> >> Dear all,
>> > >>>> >> I move to csv file because originally the date where in csv
>> > >file.
>> > >>>> >> In addition, due to the fact that, as you told me, read.csv is a
>> > >>>> >> special
>> > >>>> >> case of read.table, I prefer start to learn from the simplest
>> > >one.
>> > >>>> >> After that, I will try also the *.txt format.
>> > >>>> >>
>> > >>>> >> with read.csv, something strange happened:
>> > >>>> >>
>> > >>>> >> This us now the file:
>> > >>>> >>
>> > >>>> >> date,st1,st2,st3,
>> > >>>> >> 10/1/1998 0:00,0.6,0,0
>> > >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
>> > >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
>> > >>>> >> 10/1/1998 3:00,0,0,0.6
>> > >>>> >> 10/1/1998 4:00,0,0,0
>> > >>>> >> 10/1/1998 5:00,0,0,0
>> > >>>> >> 10/1/1998 6:00,0,0,0
>> > >>>> >> 10/1/1998 7:00,0.2,0,0
>> > >>>> >> 10/1/1998 8:00,0.6,0.2,0
>> > >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
>> > >>>> >> 10/1/1998 10:00,0,0.4,0.2
>> > >>>> >>
>> > >>>> >> When I apply:
>> > >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>> > >>>> >>
>> > >>>> >> this is the results:
>> > >>>> >>
>> > >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
>> > >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
>> > >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
>> > >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
>> > >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
>> > >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
>> > >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
>> > >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
>> > >>>> >>
>> > >>>> >> I do not understand why.
>> > >>>> >> Something wrong with date?
>> > >>>> >>
>> > >>>> >> really really thanks,
>> > >>>> >> I appreciate a lot all your helps.
>> > >>>> >>
>> > >>>> >> Diedro
>> > >>>> >>
>> > >>>> >>
>> > >>>> >> Diego
>> > >>>> >>
>> > >>>> >>
>> > >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]>
>> > >wrote:
>> > >>>> >>>
>> > >>>> >>> Or, without removing the first line
>> > >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
>> > >>>> >>>
>> > >>>> >>> Another alternative,
>> > >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>> > >>>> >>> since the dates appear to be in the default format.
>> > >>>> >>> (I generally prefer to work with datetimes in POSIXct class
>> > >rather
>> > >>>> >>> than
>> > >>>> >>> POSIXlt class)
>> > >>>> >>>
>> > >>>> >>> -Don
>> > >>>> >>>
>> > >>>> >>> --
>> > >>>> >>> Don MacQueen
>> > >>>> >>> Lawrence Livermore National Laboratory
>> > >>>> >>> 7000 East Ave., L-627
>> > >>>> >>> Livermore, CA 94550
>> > >>>> >>> 925-423-1062
>> > >>>> >>> Lab cell 925-724-7509
>> > >>>> >>>
>> > >>>> >>>
>> > >>>> >>>
>> > >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
>> > >>>> >>> <[hidden email] on behalf of
>> > >[hidden email]>
>> > >>>> >>> wrote:
>> > >>>> >>>
>> > >>>> >>>     Hi Diego,
>> > >>>> >>>     You may have to do some conversion as you have three fields
>> > >in
>> > >>>> >>> the
>> > >>>> >>>     first line using the default space separator and five
>> > >fields in
>> > >>>> >>>     subsequent lines. If the first line doesn't contain any
>> > >important
>> > >>>> >>> data
>> > >>>> >>>     you can just delete it or replace it with a meaningful
>> > >header
>> > >>>> >>> line
>> > >>>> >>>     with five fields and save the file under another name.
>> > >>>> >>>
>> > >>>> >>>     It looks as thought you have date-time as two fields. If
>> > >so, you
>> > >>>> >>> can
>> > >>>> >>>     just read the first field if you only want the date:
>> > >>>> >>>
>> > >>>> >>>     # assume you have removed the first line
>> > >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>> > >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>> > >>>> >>>
>> > >>>> >>>     If you want the date/time:
>> > >>>> >>>
>> > >>>> >>>
>> > >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>> > >>>> >>> %H:%M:%S")
>> > >>>> >>>
>> > >>>> >>>     Jim
>> > >>>> >>>
>> > >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
>> > >>>> >>> <[hidden email]> wrote:
>> > >>>> >>>     > Dear all,
>> > >>>> >>>     >
>> > >>>> >>>     > I am dealing with the reading of a *.txt file.
>> > >>>> >>>     > The txt file the following shape:
>> > >>>> >>>     >
>> > >>>> >>>     > 103001930 103001580 103001530
>> > >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
>> > >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>> > >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>> > >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
>> > >>>> >>>     > 1998-10-01 04:00:00 0 0 0
>> > >>>> >>>     > 1998-10-01 05:00:00 0 0 0
>> > >>>> >>>     > 1998-10-01 06:00:00 0 0 0
>> > >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
>> > >>>> >>>     >
>> > >>>> >>>     > If it is possible I have a coupe of questions, which will
>> > >sound
>> > >>>> >>> stupid but
>> > >>>> >>>     > they are important to me in order to understand ho R deal
>> > >with
>> > >>>> >>> file
>> > >>>> >>> or date.
>> > >>>> >>>     >
>> > >>>> >>>     > 1) Do I have to convert it to a *csv file?
>> > >>>> >>>     > 2) Can a deal with space and not ","
>> > >>>> >>>     > 3) How can I read date?
>> > >>>> >>>     >
>> > >>>> >>>     > thanks a lot to all of you,
>> > >>>> >>>     > Thanks
>> > >>>> >>>     >
>> > >>>> >>>     >
>> > >>>> >>>     > Diego
>> > >>>> >>>     >
>> > >>>> >>>     >         [[alternative HTML version deleted]]
>> > >>>> >>>     >
>> > >>>> >>>     > ______________________________________________
>> > >>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and
>> > >more,
>> > >>>> >>> see
>> > >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
>> > >>>> >>>     > PLEASE do read the posting guide
>> > >>>> >>> http://www.R-project.org/posting-guide.html
>> > >>>> >>>     > and provide commented, minimal, self-contained,
>> > >reproducible
>> > >>>> >>> code.
>> > >>>> >>>
>> > >>>> >>>     ______________________________________________
>> > >>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and
>> > >more, see
>> > >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
>> > >>>> >>>     PLEASE do read the posting guide
>> > >>>> >>> http://www.R-project.org/posting-guide.html
>> > >>>> >>>     and provide commented, minimal, self-contained,
>> > >reproducible
>> > >>>> >>> code.
>> > >>>> >>>
>> > >>>> >>>
>> > >>>> >>
>> > >>>> >
>> > >>>
>> > >>>
>> > >>
>> > >
>> > >______________________________________________
>> > >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> > >https://stat.ethz.ch/mailman/listinfo/r-help
>> > >PLEASE do read the posting guide
>> > >http://www.R-project.org/posting-guide.html
>> > >and provide commented, minimal, self-contained, reproducible code.
>> >
>> > --
>> > Sent from my phone. Please excuse my brevity.
>> >
>>
>>         [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/
>> posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

Diego Avesani
Dear all,

I have found and error in the date conversion. Now it looks like:

MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
# change date to real
MyData$date<-as.POSIXct(MyData$date, format="%*m*/%*d*/%Y %H:%M")

After that I apply the PIKAL's suggestions:

aggregate(MyData[,-1], list(day(MyData$date)), mean)

And this is the final results:

 1 -82.43636 -46.12437 -319.2710
2        2 -82.06105 -45.74184 -319.2696
3        3 -82.05527 -45.52650 -319.2416
4        4 -82.03535 -47.59191 -319.2275
5        5 -77.44928 -50.05953 -320.5798
...
31    -86.10234 -47.06247 -340.0968

However, it is not correct.
This because I have not made myself clear about my purpose. As I told you
some days ago, I have a *.csv file with hourly data from 10/21/1998
to 12/31/2016. I would like to compute the daily means. Basically, I would
like to have the mean of the hourly date for each day from 10/21/1998
to 12/31/2016 and not 31 values.

Really really thanks again,
Diego


Diego


On 2 August 2018 at 08:55, Diego Avesani <[hidden email]> wrote:

> Dear
>
> I have check the one of the line that gives me problem. I mean, which give
> NA after R processing. I think that is similar to the others:
>
> 10/12/1998 10:00,0,0,0
> 10/12/1998 11:00,0,0,0
> 10/12/1998 12:00,0,0,0
> 10/12/1998 13:00,0,0,0
> 10/12/1998 14:00,0,0,0
> 10/12/1998 15:00,0,0,0
> 10/12/1998 16:00,0,0,0
> 10/12/1998 17:00,0,0,0
>
> @jim: It seems that you suggestion is focus on reading data from the
> terminal. It is possible to apply it to a *.csv file?
>
> @Pikal: Could it be that there are some date conversion error?
>
> Thanks again,
> Diego
>
>
> Diego
>
>
> On 1 August 2018 at 17:01, jim holtman <[hidden email]> wrote:
>
>>
>> Try this:
>>
>> > library(lubridate)
>> > library(tidyverse)
>> > input <- read.csv(text = "date,str1,str2,str3
>> + 10/1/1998 0:00,0.6,0,0
>> +                   10/1/1998 1:00,0.2,0.2,0.2
>> +                   10/1/1998 2:00,0.6,0.2,0.4
>> +                   10/1/1998 3:00,0,0,0.6
>> +                   10/1/1998 4:00,0,0,0
>> +                   10/1/1998 5:00,0,0,0
>> +                   10/1/1998 6:00,0,0,0
>> +                   10/1/1998 7:00,0.2,0,0", as.is = TRUE)
>> > # convert the date and add the "day" so summarize
>> > input <- input %>%
>> +   mutate(date = mdy_hm(date),
>> +          day = floor_date(date, unit = 'day')
>> +   )
>> >
>> > by_day <- input %>%
>> +   group_by(day) %>%
>> +   summarise(m_s1 = mean(str1),
>> +             m_s2 = mean(str2),
>> +             m_s3 = mean(str3)
>> +   )
>> >
>> > by_day
>> # A tibble: 1 x 4
>>   day                  m_s1   m_s2  m_s3
>>   <dttm>              <dbl>  <dbl> <dbl>
>> 1 1998-10-01 00:00:00 0.200 0.0500 0.150
>>
>> Jim Holtman
>> *Data Munger Guru*
>>
>>
>> *What is the problem that you are trying to solve?Tell me what you want
>> to do, not how you want to do it.*
>>
>>
>> On Tue, Jul 31, 2018 at 11:54 PM Diego Avesani <[hidden email]>
>> wrote:
>>
>>> Dear all,
>>> I am sorry, I did a lot of confusion. I am sorry, I have to relax and
>>> stat
>>> all again in order to understand.
>>> If I could I would like to start again, without mixing strategy and
>>> waiting
>>> for your advice.
>>>
>>> I am really appreciate you help, really really.
>>> Here my new file, a *.csv file (buy the way, it is possible to attach it
>>> in
>>> the mailing list?)
>>>
>>> date,str1,str2,str3
>>> 10/1/1998 0:00,0.6,0,0
>>> 10/1/1998 1:00,0.2,0.2,0.2
>>> 10/1/1998 2:00,0.6,0.2,0.4
>>> 10/1/1998 3:00,0,0,0.6
>>> 10/1/1998 4:00,0,0,0
>>> 10/1/1998 5:00,0,0,0
>>> 10/1/1998 6:00,0,0,0
>>> 10/1/1998 7:00,0.2,0,0
>>>
>>>
>>> I read it as:
>>> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>>>
>>> at this point I would like to have the daily mean.
>>> What would you suggest?
>>>
>>> Really Really thanks,
>>> You are my lifesaver
>>>
>>> Thanks
>>>
>>>
>>>
>>> Diego
>>>
>>>
>>> On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]>
>>> wrote:
>>>
>>> > ... and the most common source of NA values in time data is wrong
>>> > timezones. You really need to make sure the timezone that is assumed
>>> when
>>> > the character data are converted to POSIXt agrees with the data. In
>>> most
>>> > cases the easiest way to insure this is to use
>>> >
>>> > Sys.setenv(TZ="US/Pacific")
>>> >
>>> > or whatever timezone from
>>> >
>>> > OlsonNames()
>>> >
>>> > corresponds with your data. Execute this setenv function before the
>>> > strptime or as.POSIXct() function call.
>>> >
>>> > You can use
>>> >
>>> > MyData[ is.na(MyData$datetime), ]
>>> >
>>> > to see which records are failing to convert time.
>>> >
>>> > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
>>> >
>>> > On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]>
>>> wrote:
>>> > >Hi Diego,
>>> > >I think the error is due to NA values in your data file. If I extend
>>> > >your example and run it, I get no errors:
>>> > >
>>> > >MyData<-read.table(text="103001930 103001580 103001530
>>> > >1998-10-01 00:00:00 0.6 0 0
>>> > >1998-10-01 01:00:00 0.2 0.2 0.2
>>> > >1998-10-01 02:00:00 0.6 0.2 0.4
>>> > >1998-10-01 03:00:00 0 0 0.6
>>> > >1998-10-01 04:00:00 0 0 0
>>> > >1998-10-01 05:00:00 0 0 0
>>> > >1998-10-01 06:00:00 0 0 0
>>> > >1998-10-01 07:00:00 0.2 0 0
>>> > >1998-10-01 08:00:00 0.6 0 0
>>> > >1998-10-01 09:00:00 0.2 0.2 0.2
>>> > >1998-10-01 10:00:00 0.6 0.2 0.4
>>> > >1998-10-01 11:00:00 0 0 0.6
>>> > >1998-10-01 12:00:00 0 0 0
>>> > >1998-10-01 13:00:00 0 0 0
>>> > >1998-10-01 14:00:00 0 0 0
>>> > >1998-10-01 15:00:00 0.2 0 0
>>> > >1998-10-01 16:00:00 0.6 0 0
>>> > >1998-10-01 17:00:00 0.2 0.2 0.2
>>> > >1998-10-01 18:00:00 0.6 0.2 0.4
>>> > >1998-10-01 19:00:00 0 0 0.6
>>> > >1998-10-01 20:00:00 0 0 0
>>> > >1998-10-01 21:00:00 0 0 0
>>> > >1998-10-01 22:00:00 0 0 0
>>> > >1998-10-01 23:00:00 0.2 0 0
>>> > >1998-10-02 00:00:00 0.6 0 0
>>> > >1998-10-02 01:00:00 0.2 0.2 0.2
>>> > >1998-10-02 02:00:00 0.6 0.2 0.4
>>> > >1998-10-02 03:00:00 0 0 0.6
>>> > >1998-10-02 04:00:00 0 0 0
>>> > >1998-10-02 05:00:00 0 0 0
>>> > >1998-10-02 06:00:00 0 0 0
>>> > >1998-10-02 07:00:00 0.2 0 0
>>> > >1998-10-02 08:00:00 0.6 0 0
>>> > >1998-10-02 09:00:00 0.2 0.2 0.2
>>> > >1998-10-02 10:00:00 0.6 0.2 0.4
>>> > >1998-10-02 11:00:00 0 0 0.6
>>> > >1998-10-02 12:00:00 0 0 0
>>> > >1998-10-02 13:00:00 0 0 0
>>> > >1998-10-02 14:00:00 0 0 0
>>> > >1998-10-02 15:00:00 0.2 0 0
>>> > >1998-10-02 16:00:00 0.6 0 0
>>> > >1998-10-02 17:00:00 0.2 0.2 0.2
>>> > >1998-10-02 18:00:00 0.6 0.2 0.4
>>> > >1998-10-02 19:00:00 0 0 0.6
>>> > >1998-10-02 20:00:00 0 0 0
>>> > >1998-10-02 21:00:00 0 0 0
>>> > >1998-10-02 22:00:00 0 0 0
>>> > >1998-10-02 23:00:00 0.2 0 0",
>>> > >skip=1,stringsAsFactors=FALSE)
>>> > >names(MyData)<-c("date","time","st1","st2","st3")
>>> > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
>>> > > format="%Y-%m-%d %H:%M:%S")
>>> > >MyData$datetime
>>> > >st1_daily<-by(MyData$st1,MyData$date,mean)
>>> > >st2_daily<-by(MyData$st2,MyData$date,mean)
>>> > >st3_daily<-by(MyData$st3,MyData$date,mean)
>>> > >st1_daily
>>> > >st2_daily
>>> > >st3_daily
>>> > >
>>> > >Try adding na.rm=TRUE to the "by" calls:
>>> > >
>>> > >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
>>> > >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
>>> > >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
>>> > >
>>> > >Jim
>>> > >
>>> > >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
>>> > ><[hidden email]> wrote:
>>> > >> Dear all,
>>> > >>
>>> > >> I have still problem with date.
>>> > >> Could you please tel me how to use POSIXct.
>>> > >> Indeed I have found this command:
>>> > >> timeAverage, but I am not able to convert MyDate to properly date.
>>> > >>
>>> > >> Thank a lot
>>> > >> I hope to no bother you, at least too much
>>> > >>
>>> > >>
>>> > >> Diego
>>> > >>
>>> > >>
>>> > >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]>
>>> > >wrote:
>>> > >>>
>>> > >>> Dear Jim, Dear all,
>>> > >>>
>>> > >>> thanks a lot.
>>> > >>>
>>> > >>> Unfortunately, I get the following error:
>>> > >>>
>>> > >>>
>>> > >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
>>> > >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
>>> > >925L,  :
>>> > >>>   arguments must have same length
>>> > >>>
>>> > >>>
>>> > >>> This is particularly strange. indeed, if I apply
>>> > >>>
>>> > >>>
>>> > >>> mean(MyData$str1,na.rm=TRUE)
>>> > >>>
>>> > >>>
>>> > >>> it works
>>> > >>>
>>> > >>>
>>> > >>> Sorry, I have to learn a lot.
>>> > >>> You are really boosting me
>>> > >>>
>>> > >>> Diego
>>> > >>>
>>> > >>>
>>> > >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]> wrote:
>>> > >>>>
>>> > >>>> Hi Diego,
>>> > >>>> One way you can get daily means is:
>>> > >>>>
>>> > >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
>>> > >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
>>> > >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
>>> > >>>>
>>> > >>>> Jim
>>> > >>>>
>>> > >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
>>> > ><[hidden email]>
>>> > >>>> wrote:
>>> > >>>> > Dear all,
>>> > >>>> > I have found the error, my fault. Sorry.
>>> > >>>> > There was an extra come in the headers line.
>>> > >>>> > Thanks again.
>>> > >>>> >
>>> > >>>> > If I can I would like to ask you another questions about the
>>> > >imported
>>> > >>>> > data.
>>> > >>>> > I would like to compute the daily average of the different date.
>>> > >>>> > Basically I
>>> > >>>> > have hourly data, I would like to ave the daily mean of them.
>>> > >>>> >
>>> > >>>> > Is there some special commands?
>>> > >>>> >
>>> > >>>> > Thanks a lot.
>>> > >>>> >
>>> > >>>> >
>>> > >>>> > Diego
>>> > >>>> >
>>> > >>>> >
>>> > >>>> > On 31 July 2018 at 10:40, Diego Avesani <
>>> [hidden email]>
>>> > >>>> > wrote:
>>> > >>>> >>
>>> > >>>> >> Dear all,
>>> > >>>> >> I move to csv file because originally the date where in csv
>>> > >file.
>>> > >>>> >> In addition, due to the fact that, as you told me, read.csv is
>>> a
>>> > >>>> >> special
>>> > >>>> >> case of read.table, I prefer start to learn from the simplest
>>> > >one.
>>> > >>>> >> After that, I will try also the *.txt format.
>>> > >>>> >>
>>> > >>>> >> with read.csv, something strange happened:
>>> > >>>> >>
>>> > >>>> >> This us now the file:
>>> > >>>> >>
>>> > >>>> >> date,st1,st2,st3,
>>> > >>>> >> 10/1/1998 0:00,0.6,0,0
>>> > >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
>>> > >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
>>> > >>>> >> 10/1/1998 3:00,0,0,0.6
>>> > >>>> >> 10/1/1998 4:00,0,0,0
>>> > >>>> >> 10/1/1998 5:00,0,0,0
>>> > >>>> >> 10/1/1998 6:00,0,0,0
>>> > >>>> >> 10/1/1998 7:00,0.2,0,0
>>> > >>>> >> 10/1/1998 8:00,0.6,0.2,0
>>> > >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
>>> > >>>> >> 10/1/1998 10:00,0,0.4,0.2
>>> > >>>> >>
>>> > >>>> >> When I apply:
>>> > >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>>> > >>>> >>
>>> > >>>> >> this is the results:
>>> > >>>> >>
>>> > >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
>>> > >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
>>> > >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
>>> > >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
>>> > >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
>>> > >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
>>> > >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
>>> > >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
>>> > >>>> >>
>>> > >>>> >> I do not understand why.
>>> > >>>> >> Something wrong with date?
>>> > >>>> >>
>>> > >>>> >> really really thanks,
>>> > >>>> >> I appreciate a lot all your helps.
>>> > >>>> >>
>>> > >>>> >> Diedro
>>> > >>>> >>
>>> > >>>> >>
>>> > >>>> >> Diego
>>> > >>>> >>
>>> > >>>> >>
>>> > >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]>
>>> > >wrote:
>>> > >>>> >>>
>>> > >>>> >>> Or, without removing the first line
>>> > >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE,
>>> skip=1)
>>> > >>>> >>>
>>> > >>>> >>> Another alternative,
>>> > >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>>> > >>>> >>> since the dates appear to be in the default format.
>>> > >>>> >>> (I generally prefer to work with datetimes in POSIXct class
>>> > >rather
>>> > >>>> >>> than
>>> > >>>> >>> POSIXlt class)
>>> > >>>> >>>
>>> > >>>> >>> -Don
>>> > >>>> >>>
>>> > >>>> >>> --
>>> > >>>> >>> Don MacQueen
>>> > >>>> >>> Lawrence Livermore National Laboratory
>>> > >>>> >>> 7000 East Ave., L-627
>>> > >>>> >>> Livermore, CA 94550
>>> > >>>> >>> 925-423-1062
>>> > >>>> >>> Lab cell 925-724-7509
>>> > >>>> >>>
>>> > >>>> >>>
>>> > >>>> >>>
>>> > >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
>>> > >>>> >>> <[hidden email] on behalf of
>>> > >[hidden email]>
>>> > >>>> >>> wrote:
>>> > >>>> >>>
>>> > >>>> >>>     Hi Diego,
>>> > >>>> >>>     You may have to do some conversion as you have three
>>> fields
>>> > >in
>>> > >>>> >>> the
>>> > >>>> >>>     first line using the default space separator and five
>>> > >fields in
>>> > >>>> >>>     subsequent lines. If the first line doesn't contain any
>>> > >important
>>> > >>>> >>> data
>>> > >>>> >>>     you can just delete it or replace it with a meaningful
>>> > >header
>>> > >>>> >>> line
>>> > >>>> >>>     with five fields and save the file under another name.
>>> > >>>> >>>
>>> > >>>> >>>     It looks as thought you have date-time as two fields. If
>>> > >so, you
>>> > >>>> >>> can
>>> > >>>> >>>     just read the first field if you only want the date:
>>> > >>>> >>>
>>> > >>>> >>>     # assume you have removed the first line
>>> > >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>>> > >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>>> > >>>> >>>
>>> > >>>> >>>     If you want the date/time:
>>> > >>>> >>>
>>> > >>>> >>>
>>> > >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>>> > >>>> >>> %H:%M:%S")
>>> > >>>> >>>
>>> > >>>> >>>     Jim
>>> > >>>> >>>
>>> > >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
>>> > >>>> >>> <[hidden email]> wrote:
>>> > >>>> >>>     > Dear all,
>>> > >>>> >>>     >
>>> > >>>> >>>     > I am dealing with the reading of a *.txt file.
>>> > >>>> >>>     > The txt file the following shape:
>>> > >>>> >>>     >
>>> > >>>> >>>     > 103001930 103001580 103001530
>>> > >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
>>> > >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
>>> > >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
>>> > >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
>>> > >>>> >>>     > 1998-10-01 04:00:00 0 0 0
>>> > >>>> >>>     > 1998-10-01 05:00:00 0 0 0
>>> > >>>> >>>     > 1998-10-01 06:00:00 0 0 0
>>> > >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
>>> > >>>> >>>     >
>>> > >>>> >>>     > If it is possible I have a coupe of questions, which
>>> will
>>> > >sound
>>> > >>>> >>> stupid but
>>> > >>>> >>>     > they are important to me in order to understand ho R
>>> deal
>>> > >with
>>> > >>>> >>> file
>>> > >>>> >>> or date.
>>> > >>>> >>>     >
>>> > >>>> >>>     > 1) Do I have to convert it to a *csv file?
>>> > >>>> >>>     > 2) Can a deal with space and not ","
>>> > >>>> >>>     > 3) How can I read date?
>>> > >>>> >>>     >
>>> > >>>> >>>     > thanks a lot to all of you,
>>> > >>>> >>>     > Thanks
>>> > >>>> >>>     >
>>> > >>>> >>>     >
>>> > >>>> >>>     > Diego
>>> > >>>> >>>     >
>>> > >>>> >>>     >         [[alternative HTML version deleted]]
>>> > >>>> >>>     >
>>> > >>>> >>>     > ______________________________________________
>>> > >>>> >>>     > [hidden email] mailing list -- To UNSUBSCRIBE and
>>> > >more,
>>> > >>>> >>> see
>>> > >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
>>> > >>>> >>>     > PLEASE do read the posting guide
>>> > >>>> >>> http://www.R-project.org/posting-guide.html
>>> > >>>> >>>     > and provide commented, minimal, self-contained,
>>> > >reproducible
>>> > >>>> >>> code.
>>> > >>>> >>>
>>> > >>>> >>>     ______________________________________________
>>> > >>>> >>>     [hidden email] mailing list -- To UNSUBSCRIBE and
>>> > >more, see
>>> > >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
>>> > >>>> >>>     PLEASE do read the posting guide
>>> > >>>> >>> http://www.R-project.org/posting-guide.html
>>> > >>>> >>>     and provide commented, minimal, self-contained,
>>> > >reproducible
>>> > >>>> >>> code.
>>> > >>>> >>>
>>> > >>>> >>>
>>> > >>>> >>
>>> > >>>> >
>>> > >>>
>>> > >>>
>>> > >>
>>> > >
>>> > >______________________________________________
>>> > >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> > >https://stat.ethz.ch/mailman/listinfo/r-help
>>> > >PLEASE do read the posting guide
>>> > >http://www.R-project.org/posting-guide.html
>>> > >and provide commented, minimal, self-contained, reproducible code.
>>> >
>>> > --
>>> > Sent from my phone. Please excuse my brevity.
>>> >
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posti
>>> ng-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: read txt file - date - no space

PIKAL Petr
In reply to this post by Diego Avesani
Hi

see in line (and please do not post HTML formated messages, it could be scrammbled)

From: Diego Avesani <[hidden email]>
Sent: Thursday, August 2, 2018 8:56 AM
To: jim holtman <[hidden email]>; PIKAL Petr <[hidden email]>
Cc: R mailing list <[hidden email]>
Subject: Re: [R] read txt file - date - no space

Dear

I have check the one of the line that gives me problem. I mean, which give NA after R processing. I think that is similar to the others:

You should stop **thinking** and instead do real inspection of „offending“ values.

10/12/1998 10:00,0,0,0
10/12/1998 11:00,0,0,0
10/12/1998 12:00,0,0,0
10/12/1998 13:00,0,0,0
10/12/1998 14:00,0,0,0
10/12/1998 15:00,0,0,0
10/12/1998 16:00,0,0,0
10/12/1998 17:00,0,0,0

These lines do not pose any problem with formating.

>  test<-read.table("clipboard", sep=",")
> str(test)
'data.frame':   8 obs. of  4 variables:
$ V1: Factor w/ 8 levels "10/12/1998 10:00",..: 1 2 3 4 5 6 7 8
$ V2: int  0 0 0 0 0 0 0 0
$ V3: int  0 0 0 0 0 0 0 0
$ V4: int  0 0 0 0 0 0 0 0
> as.POSIXct(test$V1, format="%d/%m/%Y %H:%M")
[1] "1998-12-10 10:00:00 CET" "1998-12-10 11:00:00 CET"
[3] "1998-12-10 12:00:00 CET" "1998-12-10 13:00:00 CET"
[5] "1998-12-10 14:00:00 CET" "1998-12-10 15:00:00 CET"
[7] "1998-12-10 16:00:00 CET" "1998-12-10 17:00:00 CET"


@jim: It seems that you suggestion is focus on reading data from the terminal. It is possible to apply it to a *.csv file?

@Pikal: Could it be that there are some date conversion error?

Well, your str(MyData) result suggest, that conversion from character to POSIX was done correctly (at least partly).

However NAs in date column you posted in second mail suggest, that some values in the input are probably formated differently and they are changed to NA during POSIX conversion.

You could check which values are problematic if instead directly changing date column to POSIX you put a new column to you data with converted POSIX values

So read your data from csv file and change date to POSIX but store it in different column of data frame.

MyData$date2 <- as.POSIXct(MyData$date, format="%d/%m/%Y %H:%M")

and check which values in your original file are formated differently.

something like
MyData$date[is.na(MyData$date2)]

However your (very basic) questions suggest, that you have only minor understanding what are R objects, how to check, inspect and manipulate them. You could do a big favour to yourself going through basic documentation as I suggested before.

Cheers
Petr

Thanks again,
Diego


Diego

On 1 August 2018 at 17:01, jim holtman <[hidden email]<mailto:[hidden email]>> wrote:

Try this:

> library(lubridate)
> library(tidyverse)
> input <- read.csv(text = "date,str1,str2,str3
+ 10/1/1998 0:00,0.6,0,0
+                   10/1/1998 1:00,0.2,0.2,0.2
+                   10/1/1998 2:00,0.6,0.2,0.4
+                   10/1/1998 3:00,0,0,0.6
+                   10/1/1998 4:00,0,0,0
+                   10/1/1998 5:00,0,0,0
+                   10/1/1998 6:00,0,0,0
+                   10/1/1998 7:00,0.2,0,0", as.is<http://as.is> = TRUE)
> # convert the date and add the "day" so summarize
> input <- input %>%
+   mutate(date = mdy_hm(date),
+          day = floor_date(date, unit = 'day')
+   )
>
> by_day <- input %>%
+   group_by(day) %>%
+   summarise(m_s1 = mean(str1),
+             m_s2 = mean(str2),
+             m_s3 = mean(str3)
+   )
>
> by_day
# A tibble: 1 x 4
  day                  m_s1   m_s2  m_s3
  <dttm>              <dbl>  <dbl> <dbl>
1 1998-10-01 00:00:00 0.200 0.0500 0.150

Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.


On Tue, Jul 31, 2018 at 11:54 PM Diego Avesani <[hidden email]<mailto:[hidden email]>> wrote:
Dear all,
I am sorry, I did a lot of confusion. I am sorry, I have to relax and stat
all again in order to understand.
If I could I would like to start again, without mixing strategy and waiting
for your advice.

I am really appreciate you help, really really.
Here my new file, a *.csv file (buy the way, it is possible to attach it in
the mailing list?)

date,str1,str2,str3
10/1/1998 0:00,0.6,0,0
10/1/1998 1:00,0.2,0.2,0.2
10/1/1998 2:00,0.6,0.2,0.4
10/1/1998 3:00,0,0,0.6
10/1/1998 4:00,0,0,0
10/1/1998 5:00,0,0,0
10/1/1998 6:00,0,0,0
10/1/1998 7:00,0.2,0,0


I read it as:
MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")

at this point I would like to have the daily mean.
What would you suggest?

Really Really thanks,
You are my lifesaver

Thanks



Diego


On 1 August 2018 at 01:01, Jeff Newmiller <[hidden email]<mailto:[hidden email]>> wrote:

> ... and the most common source of NA values in time data is wrong
> timezones. You really need to make sure the timezone that is assumed when
> the character data are converted to POSIXt agrees with the data. In most
> cases the easiest way to insure this is to use
>
> Sys.setenv(TZ="US/Pacific")
>
> or whatever timezone from
>
> OlsonNames()
>
> corresponds with your data. Execute this setenv function before the
> strptime or as.POSIXct() function call.
>
> You can use
>
> MyData[ is.na<http://is.na>(MyData$datetime), ]
>
> to see which records are failing to convert time.
>
> [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
>
> On July 31, 2018 3:04:05 PM PDT, Jim Lemon <[hidden email]<mailto:[hidden email]>> wrote:
> >Hi Diego,
> >I think the error is due to NA values in your data file. If I extend
> >your example and run it, I get no errors:
> >
> >MyData<-read.table(text="103001930 103001580 103001530
> >1998-10-01 00:00:00 0.6 0 0
> >1998-10-01 01:00:00 0.2 0.2 0.2
> >1998-10-01 02:00:00 0.6 0.2 0.4
> >1998-10-01 03:00:00 0 0 0.6
> >1998-10-01 04:00:00 0 0 0
> >1998-10-01 05:00:00 0 0 0
> >1998-10-01 06:00:00 0 0 0
> >1998-10-01 07:00:00 0.2 0 0
> >1998-10-01 08:00:00 0.6 0 0
> >1998-10-01 09:00:00 0.2 0.2 0.2
> >1998-10-01 10:00:00 0.6 0.2 0.4
> >1998-10-01 11:00:00 0 0 0.6
> >1998-10-01 12:00:00 0 0 0
> >1998-10-01 13:00:00 0 0 0
> >1998-10-01 14:00:00 0 0 0
> >1998-10-01 15:00:00 0.2 0 0
> >1998-10-01 16:00:00 0.6 0 0
> >1998-10-01 17:00:00 0.2 0.2 0.2
> >1998-10-01 18:00:00 0.6 0.2 0.4
> >1998-10-01 19:00:00 0 0 0.6
> >1998-10-01 20:00:00 0 0 0
> >1998-10-01 21:00:00 0 0 0
> >1998-10-01 22:00:00 0 0 0
> >1998-10-01 23:00:00 0.2 0 0
> >1998-10-02 00:00:00 0.6 0 0
> >1998-10-02 01:00:00 0.2 0.2 0.2
> >1998-10-02 02:00:00 0.6 0.2 0.4
> >1998-10-02 03:00:00 0 0 0.6
> >1998-10-02 04:00:00 0 0 0
> >1998-10-02 05:00:00 0 0 0
> >1998-10-02 06:00:00 0 0 0
> >1998-10-02 07:00:00 0.2 0 0
> >1998-10-02 08:00:00 0.6 0 0
> >1998-10-02 09:00:00 0.2 0.2 0.2
> >1998-10-02 10:00:00 0.6 0.2 0.4
> >1998-10-02 11:00:00 0 0 0.6
> >1998-10-02 12:00:00 0 0 0
> >1998-10-02 13:00:00 0 0 0
> >1998-10-02 14:00:00 0 0 0
> >1998-10-02 15:00:00 0.2 0 0
> >1998-10-02 16:00:00 0.6 0 0
> >1998-10-02 17:00:00 0.2 0.2 0.2
> >1998-10-02 18:00:00 0.6 0.2 0.4
> >1998-10-02 19:00:00 0 0 0.6
> >1998-10-02 20:00:00 0 0 0
> >1998-10-02 21:00:00 0 0 0
> >1998-10-02 22:00:00 0 0 0
> >1998-10-02 23:00:00 0.2 0 0",
> >skip=1,stringsAsFactors=FALSE)
> >names(MyData)<-c("date","time","st1","st2","st3")
> >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > format="%Y-%m-%d %H:%M:%S")
> >MyData$datetime
> >st1_daily<-by(MyData$st1,MyData$date,mean)
> >st2_daily<-by(MyData$st2,MyData$date,mean)
> >st3_daily<-by(MyData$st3,MyData$date,mean)
> >st1_daily
> >st2_daily
> >st3_daily
> >
> >Try adding na.rm=TRUE to the "by" calls:
> >
> >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
> >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
> >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
> >
> >Jim
> >
> >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
> ><[hidden email]<mailto:[hidden email]>> wrote:
> >> Dear all,
> >>
> >> I have still problem with date.
> >> Could you please tel me how to use POSIXct.
> >> Indeed I have found this command:
> >> timeAverage, but I am not able to convert MyDate to properly date.
> >>
> >> Thank a lot
> >> I hope to no bother you, at least too much
> >>
> >>
> >> Diego
> >>
> >>
> >> On 31 July 2018 at 11:12, Diego Avesani <[hidden email]<mailto:[hidden email]>>
> >wrote:
> >>>
> >>> Dear Jim, Dear all,
> >>>
> >>> thanks a lot.
> >>>
> >>> Unfortunately, I get the following error:
> >>>
> >>>
> >>>  st1_daily<-by(MyData$st1,MyData$date,mean)
> >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
> >925L,  :
> >>>   arguments must have same length
> >>>
> >>>
> >>> This is particularly strange. indeed, if I apply
> >>>
> >>>
> >>> mean(MyData$str1,na.rm=TRUE)
> >>>
> >>>
> >>> it works
> >>>
> >>>
> >>> Sorry, I have to learn a lot.
> >>> You are really boosting me
> >>>
> >>> Diego
> >>>
> >>>
> >>> On 31 July 2018 at 11:02, Jim Lemon <[hidden email]<mailto:[hidden email]>> wrote:
> >>>>
> >>>> Hi Diego,
> >>>> One way you can get daily means is:
> >>>>
> >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
> >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
> >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
> >>>>
> >>>> Jim
> >>>>
> >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
> ><[hidden email]<mailto:[hidden email]>>
> >>>> wrote:
> >>>> > Dear all,
> >>>> > I have found the error, my fault. Sorry.
> >>>> > There was an extra come in the headers line.
> >>>> > Thanks again.
> >>>> >
> >>>> > If I can I would like to ask you another questions about the
> >imported
> >>>> > data.
> >>>> > I would like to compute the daily average of the different date.
> >>>> > Basically I
> >>>> > have hourly data, I would like to ave the daily mean of them.
> >>>> >
> >>>> > Is there some special commands?
> >>>> >
> >>>> > Thanks a lot.
> >>>> >
> >>>> >
> >>>> > Diego
> >>>> >
> >>>> >
> >>>> > On 31 July 2018 at 10:40, Diego Avesani <[hidden email]<mailto:[hidden email]>>
> >>>> > wrote:
> >>>> >>
> >>>> >> Dear all,
> >>>> >> I move to csv file because originally the date where in csv
> >file.
> >>>> >> In addition, due to the fact that, as you told me, read.csv is a
> >>>> >> special
> >>>> >> case of read.table, I prefer start to learn from the simplest
> >one.
> >>>> >> After that, I will try also the *.txt format.
> >>>> >>
> >>>> >> with read.csv, something strange happened:
> >>>> >>
> >>>> >> This us now the file:
> >>>> >>
> >>>> >> date,st1,st2,st3,
> >>>> >> 10/1/1998 0:00,0.6,0,0
> >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
> >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
> >>>> >> 10/1/1998 3:00,0,0,0.6
> >>>> >> 10/1/1998 4:00,0,0,0
> >>>> >> 10/1/1998 5:00,0,0,0
> >>>> >> 10/1/1998 6:00,0,0,0
> >>>> >> 10/1/1998 7:00,0.2,0,0
> >>>> >> 10/1/1998 8:00,0.6,0.2,0
> >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
> >>>> >> 10/1/1998 10:00,0,0.4,0.2
> >>>> >>
> >>>> >> When I apply:
> >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> >>>> >>
> >>>> >> this is the results:
> >>>> >>
> >>>> >> 10/1/1998 0:00    0.6    0.00    0.0 NA
> >>>> >> 2        10/1/1998 1:00    0.2    0.20    0.2 NA
> >>>> >> 3        10/1/1998 2:00    0.6    0.20    0.4 NA
> >>>> >> 4        10/1/1998 3:00    0.0    0.00    0.6 NA
> >>>> >> 5        10/1/1998 4:00    0.0    0.00    0.0 NA
> >>>> >> 6        10/1/1998 5:00    0.0    0.00    0.0 NA
> >>>> >> 7        10/1/1998 6:00    0.0    0.00    0.0 NA
> >>>> >> 8        10/1/1998 7:00    0.2    0.00    0.0 NA
> >>>> >>
> >>>> >> I do not understand why.
> >>>> >> Something wrong with date?
> >>>> >>
> >>>> >> really really thanks,
> >>>> >> I appreciate a lot all your helps.
> >>>> >>
> >>>> >> Diedro
> >>>> >>
> >>>> >>
> >>>> >> Diego
> >>>> >>
> >>>> >>
> >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <[hidden email]<mailto:[hidden email]>>
> >wrote:
> >>>> >>>
> >>>> >>> Or, without removing the first line
> >>>> >>>   dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
> >>>> >>>
> >>>> >>> Another alternative,
> >>>> >>>    dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> >>>> >>> since the dates appear to be in the default format.
> >>>> >>> (I generally prefer to work with datetimes in POSIXct class
> >rather
> >>>> >>> than
> >>>> >>> POSIXlt class)
> >>>> >>>
> >>>> >>> -Don
> >>>> >>>
> >>>> >>> --
> >>>> >>> Don MacQueen
> >>>> >>> Lawrence Livermore National Laboratory
> >>>> >>> 7000 East Ave., L-627
> >>>> >>> Livermore, CA 94550
> >>>> >>> 925-423-1062
> >>>> >>> Lab cell 925-724-7509
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
> >>>> >>> <[hidden email]<mailto:[hidden email]> on behalf of
> >[hidden email]<mailto:[hidden email]>>
> >>>> >>> wrote:
> >>>> >>>
> >>>> >>>     Hi Diego,
> >>>> >>>     You may have to do some conversion as you have three fields
> >in
> >>>> >>> the
> >>>> >>>     first line using the default space separator and five
> >fields in
> >>>> >>>     subsequent lines. If the first line doesn't contain any
> >important
> >>>> >>> data
> >>>> >>>     you can just delete it or replace it with a meaningful
> >header
> >>>> >>> line
> >>>> >>>     with five fields and save the file under another name.
> >>>> >>>
> >>>> >>>     It looks as thought you have date-time as two fields. If
> >so, you
> >>>> >>> can
> >>>> >>>     just read the first field if you only want the date:
> >>>> >>>
> >>>> >>>     # assume you have removed the first line
> >>>> >>>     dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
> >>>> >>>     dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
> >>>> >>>
> >>>> >>>     If you want the date/time:
> >>>> >>>
> >>>> >>>
> >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> >>>> >>> %H:%M:%S")
> >>>> >>>
> >>>> >>>     Jim
> >>>> >>>
> >>>> >>>     On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
> >>>> >>> <[hidden email]<mailto:[hidden email]>> wrote:
> >>>> >>>     > Dear all,
> >>>> >>>     >
> >>>> >>>     > I am dealing with the reading of a *.txt file.
> >>>> >>>     > The txt file the following shape:
> >>>> >>>     >
> >>>> >>>     > 103001930 103001580 103001530
> >>>> >>>     > 1998-10-01 00:00:00 0.6 0 0
> >>>> >>>     > 1998-10-01 01:00:00 0.2 0.2 0.2
> >>>> >>>     > 1998-10-01 02:00:00 0.6 0.2 0.4
> >>>> >>>     > 1998-10-01 03:00:00 0 0 0.6
> >>>> >>>     > 1998-10-01 04:00:00 0 0 0
> >>>> >>>     > 1998-10-01 05:00:00 0 0 0
> >>>> >>>     > 1998-10-01 06:00:00 0 0 0
> >>>> >>>     > 1998-10-01 07:00:00 0.2 0 0
> >>>> >>>     >
> >>>> >>>     > If it is possible I have a coupe of questions, which will
> >sound
> >>>> >>> stupid but
> >>>> >>>     > they are important to me in order to understand ho R deal
> >with
> >>>> >>> file
> >>>> >>> or date.
> >>>> >>>     >
> >>>> >>>     > 1) Do I have to convert it to a *csv file?
> >>>> >>>     > 2) Can a deal with space and not ","
> >>>> >>>     > 3) How can I read date?
> >>>> >>>     >
> >>>> >>>     > thanks a lot to all of you,
> >>>> >>>     > Thanks
> >>>> >>>     >
> >>>> >>>     >
> >>>> >>>     > Diego
> >>>> >>>     >
> >>>> >>>     >         [[alternative HTML version deleted]]
> >>>> >>>     >
> >>>> >>>     > ______________________________________________
> >>>> >>>     > [hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and
> >more,
> >>>> >>> see
> >>>> >>>     > https://stat.ethz.ch/mailman/listinfo/r-help
> >>>> >>>     > PLEASE do read the posting guide
> >>>> >>> http://www.R-project.org/posting-guide.html
> >>>> >>>     > and provide commented, minimal, self-contained,
> >reproducible
> >>>> >>> code.
> >>>> >>>
> >>>> >>>     ______________________________________________
> >>>> >>>     [hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and
> >more, see
> >>>> >>>     https://stat.ethz.ch/mailman/listinfo/r-help
> >>>> >>>     PLEASE do read the posting guide
> >>>> >>> http://www.R-project.org/posting-guide.html
> >>>> >>>     and provide commented, minimal, self-contained,
> >reproducible
> >>>> >>> code.
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >
> >>>
> >>>
> >>
> >
> >______________________________________________
> >[hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> --
> Sent from my phone. Please excuse my brevity.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Osobní údaje: Informace o zpracování a ochraně osobních údajů obchodních partnerů PRECHEZA a.s. jsou zveřejněny na: https://www.precheza.cz/zasady-ochrany-osobnich-udaju/ | Information about processing and protection of business partner’s personal data are available on website: https://www.precheza.cz/en/personal-data-protection-principles/
Důvěrnost: Tento e-mail a jakékoliv k němu připojené dokumenty jsou důvěrné a podléhají tomuto právně závaznému prohláąení o vyloučení odpovědnosti: https://www.precheza.cz/01-dovetek/ | This email and any documents attached to it may be confidential and are subject to the legally binding disclaimer: https://www.precheza.cz/en/01-disclaimer/


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
12