NOAA .grb2 files

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

NOAA .grb2 files

Philip
I’m trying to download NOAA Rapid Refresh model weather data but I keep getting the error message below.  Do I just need a computer with more memory?

Philip

***********************************************************************************************************************************************
Error in paste(gsub("\"", "", csv.str), collapse = ",") :
  could not allocate memory (994 Mb) in C function 'R_AllocStringBuffer'
        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: NOAA .grb2 files

Rich Shepard
On Fri, 4 Sep 2020, Philip wrote:

> I’m trying to download NOAA Rapid Refresh model weather data but I keep
> getting the error message below. Do I just need a computer with more
> memory?

Philip,

And how much available memory do you have on that host?

Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: NOAA .grb2 files

R help mailing list-2
In reply to this post by Philip
Hi Philip:

It would help if you gave the complete script you are trying to run,  and the name of the file.  

for those unfamiliar with all this,  Philip has already downloaded the grib2 file,  either using rNOMADS or directly from the NOAA website,  and the function he is calling reads the data from that grib2 file by wrapping a system call to a C-program called wgrib2.  And if you don't know about grib2 files,  they are highly compressed files of fields,  using bit-packing to do the compression. Forecast grib2 files usually have a plethora of fields,  for example since I don't know exactly which  NOAA Rapid Refresh model files he obtain,  looking at the inventory of one possible such file,  see:

https://www.nco.ncep.noaa.gov/pmb/products/rap/rap.t00z.awp252pgrbf00.grib2.shtml

Usually given this,  only one or a couple of fields at a time are unpacked,  unless you have a large computer.  Since no example script was given,  it is impossible to tell if he is trying to read in the entire grib2 file or what.  And each expanded array in R will be much much bigger than the equivalent in grib2.

-Roy



> On Sep 4, 2020, at 3:32 PM, Philip <[hidden email]> wrote:
>
> I’m trying to download NOAA Rapid Refresh model weather data but I keep getting the error message below.  Do I just need a computer with more memory?
>
> Philip
>
> ***********************************************************************************************************************************************
> Error in paste(gsub("\"", "", csv.str), collapse = ",") :
>  could not allocate memory (994 Mb) in C function 'R_AllocStringBuffer'
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: [hidden email] www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.