Read fst files

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Read fst files

reichmaj
R-Help Forum

 

Anyone know why the following line of code would error out:  myObject <-
read_fst(unz("Dataset.zip", filename = "filename.fst"))

 

Error: Incomplete expression: filename <- read_fst(unz("Dataset.zip",
filename = "filename.fst")

 

I often use similar code with *.csv files in a zipped folder. For example:
myObject <- read.csv(unz("Dataset.zip", filename = "filename.csv")), which
works just fine.

 

Jeff Reichman

 


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

Eric Berger
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

reichmaj
Eric

 

Typo on my point.

 

setwd("C:/Users/reichmaj/Documents/My_Reference_Library /Regression")

myObject <- read_fst(unz("Dataset.zip", filename = "myFile.fst")) # read fst file

 

Error in path.expand(path) : invalid 'path' argument

 

So then I tried

 

myObject <- read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library /Regression /Dataset.zip", filename = "myFile.fst"))

 

Error in path.expand(path) : invalid 'path' argument

 

Error in the path??

 

Because this works just fine

 

myObject <- read.csv(unz("C:/Users/reichmaj/Documents/My_Reference_Library /Regression /Dataset.zip", filename = "myFile.csv"))

 

My only though is I can’t use the two function s together when dealing with fst files ??

 

From: Eric Berger <[hidden email]>
Sent: Wednesday, June 9, 2021 3:50 AM
To: [hidden email]
Cc: R mailing list <[hidden email]>
Subject: Re: [R] Read fst files

 

You are missing the second closing parenthesis. This is what the error message is telling you.

 

 

On Wed, Jun 9, 2021 at 2:44 AM Jeff Reichman <[hidden email] <mailto:[hidden email]> > wrote:

R-Help Forum



Anyone know why the following line of code would error out:  myObject <-
read_fst(unz("Dataset.zip", filename = "filename.fst"))



Error: Incomplete expression: filename <- read_fst(unz("Dataset.zip",
filename = "filename.fst")



I often use similar code with *.csv files in a zipped folder. For example:
myObject <- read.csv(unz("Dataset.zip", filename = "filename.csv")), which
works just fine.



Jeff Reichman




        [[alternative HTML version deleted]]

______________________________________________
[hidden email] <mailto:[hidden email]>  mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

Eric Berger
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

Duncan Murdoch-2
In reply to this post by reichmaj
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

reichmaj
Duncan

Yea that will work. It appears to be related to setting my working dir, for what ever reason neither seem to work
(1) knitr::opts_knit$set(root.dir ="~/My_Reference_Library/Regression") # from R Notebook or
(2) setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") # from R chunk

So it appears I can either (as you suggested) use two steps or combine but I need to enter the full path. Why other file types don't seem to need the full path ....?????

myObject <- read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip", filename = "myFile.fst"))

Thank you. I guess just one of those R things

Jeff



-----Original Message-----
From: Duncan Murdoch <[hidden email]>
Sent: Wednesday, June 9, 2021 7:27 AM
To: [hidden email]; 'Eric Berger' <[hidden email]>
Cc: 'R mailing list' <[hidden email]>
Subject: Re: [R] Read fst files

It looks as though read_fst wants a filename, not a connection.

You should do it in two steps:

  unzip("Dataset.zip", files = "myFile.fst")
  myObject <- read_fst("myFile.fst")

This is obviously untested; you didn't even say what package read_fst() comes from.

Duncan Murdoch

On 09/06/2021 8:18 a.m., Jeff Reichman wrote:

> Eric
>
>  
>
> Typo on my point.
>
>  
>
> setwd("C:/Users/reichmaj/Documents/My_Reference_Library /Regression")
>
> myObject <- read_fst(unz("Dataset.zip", filename = "myFile.fst")) #
> read fst file
>
>  
>
> Error in path.expand(path) : invalid 'path' argument
>
>  
>
> So then I tried
>
>  
>
> myObject <-
> read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library
> /Regression /Dataset.zip", filename = "myFile.fst"))
>
>  
>
> Error in path.expand(path) : invalid 'path' argument
>
>  
>
> Error in the path??
>
>  
>
> Because this works just fine
>
>  
>
> myObject <-
> read.csv(unz("C:/Users/reichmaj/Documents/My_Reference_Library
> /Regression /Dataset.zip", filename = "myFile.csv"))
>
>  
>
> My only though is I can’t use the two function s together when dealing with fst files ??
>
>  
>
> From: Eric Berger <[hidden email]>
> Sent: Wednesday, June 9, 2021 3:50 AM
> To: [hidden email]
> Cc: R mailing list <[hidden email]>
> Subject: Re: [R] Read fst files
>
>  
>
> You are missing the second closing parenthesis. This is what the error message is telling you.
>
>  
>
>  
>
> On Wed, Jun 9, 2021 at 2:44 AM Jeff Reichman <[hidden email] <mailto:[hidden email]> > wrote:
>
> R-Help Forum
>
>
>
> Anyone know why the following line of code would error out:  myObject
> <- read_fst(unz("Dataset.zip", filename = "filename.fst"))
>
>
>
> Error: Incomplete expression: filename <- read_fst(unz("Dataset.zip",
> filename = "filename.fst")
>
>
>
> I often use similar code with *.csv files in a zipped folder. For example:
> myObject <- read.csv(unz("Dataset.zip", filename = "filename.csv")),
> which works just fine.
>
>
>
> Jeff Reichman
>
>
>
>
>          [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] <mailto:[hidden email]>  mailing list -- To
> UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

Duncan Murdoch-2
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

Jan van der LAan-2
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

reichmaj
Jan

Makes sense. Its just that I often receive  large zip files that contain a variety of file types.

Jef

-----Original Message-----
From: R-help <[hidden email]> On Behalf Of Jan van der Laan
Sent: Wednesday, June 9, 2021 12:56 PM
To: [hidden email]
Subject: Re: [R] Read fst files



read_fst is from the package fst. The fileformat fst uses is a binary
format designed to be fast readable. It is a column  oriented format and
compressed. So, to be able to work fst needs access to the file itself
and wont accept a file connection as functions like read.table an
variants accept.

Also, because it is a binary compressed format using a compression
method that is fast to read, compressing also to zip seems to defeat the
purpose of fst.

HTH,
Jan


On 09-06-2021 15:28, Duncan Murdoch wrote:

> On 09/06/2021 9:12 a.m., Jeff Reichman wrote:
>> Duncan
>>
>> Yea that will work. It appears to be related to setting my working
>> dir, for what ever reason neither seem to work
>> (1) knitr::opts_knit$set(root.dir
>> ="~/My_Reference_Library/Regression") # from R Notebook or
>> (2)
>> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") #
>> from R chunk
>>
>> So it appears I can either (as you suggested) use two steps or combine
>> but I need to enter the full path. Why other file types don't seem to
>> need the full path ....?????
>
> You need to read the documentation for read_fst() to find what it needs.
>   If it doesn't explain this, then you should report the issue to its
> author.
>
>>
>> myObject <-
>> read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip",
>> filename = "myFile.fst"))
>>
>> Thank you. I guess just one of those R things
>
> No, it's a read_fst() thing.
>
> Duncan Murdoch
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

Jeff Newmiller
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

Bill Dunlap-2
In reply to this post by reichmaj
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

Duncan Murdoch-2
In reply to this post by Jan van der LAan-2
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Read fst files

reichmaj
In reply to this post by Bill Dunlap-2
Bill

 

So I understand that’s just unzipping the file to a temporary dir which then would allow read_fst to access the file directly .

 

Jeff

 

From: Bill Dunlap <[hidden email]>
Sent: Wednesday, June 9, 2021 1:43 PM
To: [hidden email]
Cc: Jan van der Laan <[hidden email]>; [hidden email]
Subject: Re: [R] Read fst files

 

Try using unzip(zipfile, files="desiredFile", exdir=tf<-tempfile()), not unz(zipfile, "desiredFile"), to copy the desired file from the zip file to a temporary location and use read_fst(tf) to read the desired file.

 

-Bill

 

On Wed, Jun 9, 2021 at 11:27 AM Jeff Reichman <[hidden email] <mailto:[hidden email]> > wrote:

Jan

Makes sense. Its just that I often receive  large zip files that contain a variety of file types.

Jef

-----Original Message-----
From: R-help <[hidden email] <mailto:[hidden email]> > On Behalf Of Jan van der Laan
Sent: Wednesday, June 9, 2021 12:56 PM
To: [hidden email] <mailto:[hidden email]>
Subject: Re: [R] Read fst files



read_fst is from the package fst. The fileformat fst uses is a binary
format designed to be fast readable. It is a column  oriented format and
compressed. So, to be able to work fst needs access to the file itself
and wont accept a file connection as functions like read.table an
variants accept.

Also, because it is a binary compressed format using a compression
method that is fast to read, compressing also to zip seems to defeat the
purpose of fst.

HTH,
Jan


On 09-06-2021 15:28, Duncan Murdoch wrote:

> On 09/06/2021 9:12 a.m., Jeff Reichman wrote:
>> Duncan
>>
>> Yea that will work. It appears to be related to setting my working
>> dir, for what ever reason neither seem to work
>> (1) knitr::opts_knit$set(root.dir
>> ="~/My_Reference_Library/Regression") # from R Notebook or
>> (2)
>> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") #
>> from R chunk
>>
>> So it appears I can either (as you suggested) use two steps or combine
>> but I need to enter the full path. Why other file types don't seem to
>> need the full path ....?????
>
> You need to read the documentation for read_fst() to find what it needs.
>   If it doesn't explain this, then you should report the issue to its
> author.
>
>>
>> myObject <-
>> read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip",
>> filename = "myFile.fst"))
>>
>> Thank you. I guess just one of those R things
>
> No, it's a read_fst() thing.
>
> Duncan Murdoch
>
> ______________________________________________
> [hidden email] <mailto:[hidden email]>  mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] <mailto:[hidden email]>  mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] <mailto:[hidden email]>  mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.