Understanding read.csv error message

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Understanding read.csv error message

Rich Shepard
   I have a data file, 'precip_projected.csv,' that starts like this:

name,easting,northing,elev,sampdate,prcp
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-01,0.59
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-02,0.08
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-03,0.1
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-04,0
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-05,0
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-06,0.02
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-07,0.05
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-08,0.1
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-09,0
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-10,0.02

   There are a bunch of NAs in the data file.

   The command to read it produces an error:

rainfall <- read.csv('../data/precipitation/precip_projected.csv', header = T, sep = ',', as.is = T)

Error in `$<-.data.frame`(`*tmp*`, date, value = numeric(0)) :
   replacement has 0 rows, data has 113569

   Is the error generated by finding a date that looks like the number zero
or by a prcp value of zero?

   BTW, I get the same error if I specify stringsAsFactors = F.

TIA,

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Understanding read.csv error message

R help mailing list-2
Hi Rich,

It's not obvious what would be causing that error from read.csv. But here's what I would probably try:

Add quote='"" to your arguments. The default is to use surround text strings with double quotes, but your file doesn't.

Copy the first few rows into another file and try it. If it succeeds, that would suggest something later on in the file is causing the problem.

The argument sep=','   is redundant for read.csv. In other words, it sets sep for you. I'd try switching to the more general read.table.

Are the NAs in the file indicated by NA between a pair of commas? Or do you have successive commas with nothing between them for NA? Not sure what difference it will make, but it might affect what args you pass to read.table.

Are you absolutely sure there are never any commas in the name?

-Don

--
Don MacQueen
Lawrence Livermore National Laboratory
7000 East Ave., L-627
Livermore, CA 94550
925-423-1062
Lab cell 925-724-7509
 
 

On 8/17/18, 3:08 PM, "R-help on behalf of Rich Shepard" <[hidden email] on behalf of [hidden email]> wrote:

       I have a data file, 'precip_projected.csv,' that starts like this:
   
    name,easting,northing,elev,sampdate,prcp
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-01,0.59
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-02,0.08
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-03,0.1
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-04,0
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-05,0
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-06,0.02
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-07,0.05
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-08,0.1
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-09,0
    Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-10,0.02
   
       There are a bunch of NAs in the data file.
   
       The command to read it produces an error:
   
    rainfall <- read.csv('../data/precipitation/precip_projected.csv', header = T, sep = ',', as.is = T)
   
    Error in `$<-.data.frame`(`*tmp*`, date, value = numeric(0)) :
       replacement has 0 rows, data has 113569
   
       Is the error generated by finding a date that looks like the number zero
    or by a prcp value of zero?
   
       BTW, I get the same error if I specify stringsAsFactors = F.
   
    TIA,
   
    Rich
   
    ______________________________________________
    [hidden email] mailing list -- To UNSUBSCRIBE and more, see
    https://stat.ethz.ch/mailman/listinfo/r-help
    PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
    and provide commented, minimal, self-contained, reproducible code.
   

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Understanding read.csv error message

R help mailing list-2
small typo in previous: should be
  quote=""
(I left behind a single quote by mistake)

--
Don MacQueen
Lawrence Livermore National Laboratory
7000 East Ave., L-627
Livermore, CA 94550
925-423-1062
Lab cell 925-724-7509
 
 

On 8/17/18, 5:03 PM, "R-help on behalf of MacQueen, Don via R-help" <[hidden email] on behalf of [hidden email]> wrote:

    Hi Rich,
   
    It's not obvious what would be causing that error from read.csv. But here's what I would probably try:
   
    Add quote='"" to your arguments. The default is to use surround text strings with double quotes, but your file doesn't.
   
    Copy the first few rows into another file and try it. If it succeeds, that would suggest something later on in the file is causing the problem.
   
    The argument sep=','   is redundant for read.csv. In other words, it sets sep for you. I'd try switching to the more general read.table.
   
    Are the NAs in the file indicated by NA between a pair of commas? Or do you have successive commas with nothing between them for NA? Not sure what difference it will make, but it might affect what args you pass to read.table.
   
    Are you absolutely sure there are never any commas in the name?
   
    -Don
   
    --
    Don MacQueen
    Lawrence Livermore National Laboratory
    7000 East Ave., L-627
    Livermore, CA 94550
    925-423-1062
    Lab cell 925-724-7509
     
     
   
    On 8/17/18, 3:08 PM, "R-help on behalf of Rich Shepard" <[hidden email] on behalf of [hidden email]> wrote:
   
           I have a data file, 'precip_projected.csv,' that starts like this:
       
        name,easting,northing,elev,sampdate,prcp
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-01,0.59
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-02,0.08
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-03,0.1
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-04,0
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-05,0
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-06,0.02
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-07,0.05
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-08,0.1
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-09,0
        Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-10,0.02
       
           There are a bunch of NAs in the data file.
       
           The command to read it produces an error:
       
        rainfall <- read.csv('../data/precipitation/precip_projected.csv', header = T, sep = ',', as.is = T)
       
        Error in `$<-.data.frame`(`*tmp*`, date, value = numeric(0)) :
           replacement has 0 rows, data has 113569
       
           Is the error generated by finding a date that looks like the number zero
        or by a prcp value of zero?
       
           BTW, I get the same error if I specify stringsAsFactors = F.
       
        TIA,
       
        Rich
       
        ______________________________________________
        [hidden email] mailing list -- To UNSUBSCRIBE and more, see
        https://stat.ethz.ch/mailman/listinfo/r-help
        PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
        and provide commented, minimal, self-contained, reproducible code.
       
   
    ______________________________________________
    [hidden email] mailing list -- To UNSUBSCRIBE and more, see
    https://stat.ethz.ch/mailman/listinfo/r-help
    PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
    and provide commented, minimal, self-contained, reproducible code.
   

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Understanding read.csv error message

Peter Dalgaard-2
In reply to this post by Rich Shepard
What Don said, and also notice that the error is not about anything having value 0, it is about replacing something with something of _length_ 0. It is not obvious where that happens, sometimes a traceback() can give a clue, but probably Don is right that the issue is that there is something not quite CSV in the file.

One further idea is to read using colClasses="character" and see if that actually gives you 6 columns and then afterwards try and convert each column to the appropriate type.

-pd

> On 18 Aug 2018, at 00:08 , Rich Shepard <[hidden email]> wrote:
>
>  I have a data file, 'precip_projected.csv,' that starts like this:
>
> name,easting,northing,elev,sampdate,prcp
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-01,0.59
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-02,0.08
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-03,0.1
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-04,0
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-05,0
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-06,0.02
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-07,0.05
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-08,0.1
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-09,0
> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-10,0.02
>
>  There are a bunch of NAs in the data file.
>
>  The command to read it produces an error:
>
> rainfall <- read.csv('../data/precipitation/precip_projected.csv', header = T, sep = ',', as.is = T)
>
> Error in `$<-.data.frame`(`*tmp*`, date, value = numeric(0)) :
>  replacement has 0 rows, data has 113569
>
>  Is the error generated by finding a date that looks like the number zero
> or by a prcp value of zero?
>
>  BTW, I get the same error if I specify stringsAsFactors = F.
>
> TIA,
>
> Rich
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Understanding read.csv error message

Ben Tupper-2
Hi,

I would be tempted, as a start, to read the entire file in as rows of text, split each line by the expected delimiter, and then count the number of elements each split line yields.  Once you know each row splits into the expected number of

txt <- "name,easting,northing,elev,sampdate,prcp
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-01,0.59
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-02,0.08
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-03,0.1
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-04,0
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-05,0
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-06,0.02
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-07,0.05
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-08,0.1
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-09,0
Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-10,0.02"

txtCon <- textConnection(txt)
x <- readLines(txtCon)
close(txtCon)

n <- sapply(strsplit(x, ",", fixed = TRUE), length)
table(n)

If any have a different length then investigate that/those line(s).  If they all have the same length then it likely isn't about the delimiter.

Cheers,
Ben


 

> On Aug 18, 2018, at 5:05 AM, peter dalgaard <[hidden email]> wrote:
>
> What Don said, and also notice that the error is not about anything having value 0, it is about replacing something with something of _length_ 0. It is not obvious where that happens, sometimes a traceback() can give a clue, but probably Don is right that the issue is that there is something not quite CSV in the file.
>
> One further idea is to read using colClasses="character" and see if that actually gives you 6 columns and then afterwards try and convert each column to the appropriate type.
>
> -pd
>
>> On 18 Aug 2018, at 00:08 , Rich Shepard <[hidden email]> wrote:
>>
>> I have a data file, 'precip_projected.csv,' that starts like this:
>>
>> name,easting,northing,elev,sampdate,prcp
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-01,0.59
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-02,0.08
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-03,0.1
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-04,0
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-05,0
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-06,0.02
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-07,0.05
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-08,0.1
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-09,0
>> Headworks Portland Water,2370575.38427211,199337.634652112,228,2005-01-10,0.02
>>
>> There are a bunch of NAs in the data file.
>>
>> The command to read it produces an error:
>>
>> rainfall <- read.csv('../data/precipitation/precip_projected.csv', header = T, sep = ',', as.is = T)
>>
>> Error in `$<-.data.frame`(`*tmp*`, date, value = numeric(0)) :
>> replacement has 0 rows, data has 113569
>>
>> Is the error generated by finding a date that looks like the number zero
>> or by a prcp value of zero?
>>
>> BTW, I get the same error if I specify stringsAsFactors = F.
>>
>> TIA,
>>
>> Rich
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: [hidden email]  Priv: [hidden email]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Ecological Forecasting: https://eco.bigelow.org/






        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Understanding read.csv error message [FIXED]

Rich Shepard
In reply to this post by R help mailing list-2
On Sat, 18 Aug 2018, MacQueen, Don wrote:

> It's not obvious what would be causing that error from read.csv. But
> here's what I would probably try:

> I'd try switching to the more general read.table.

Don/Peter,

   I found the problem: it was in the following line in the script which
referenced 'date' rather than 'sampdate'.

   Single or double quotes make no difference within the script.

   I have a vague recollection from long ago that read.table() is a better
choice than read.csv(). I don't recall the details why, but I did change the
function to read.table(), fixed the column name, and have only the labeling
of the xyploy() left to fix.

Thanks very much, both of you,

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.