Error exporting dataframe from Julia to R with Feather

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Error exporting dataframe from Julia to R with Feather

Luigi
Hello,

since I am encountering a lot of problems exporting dataframes from
julia to R (there is always something wrong with the formatting,
probably a missing quote) so I am trying to use Feather to do the job.

I have installed Feather in Julia with `pkg.add("Feather")` and
imported it with `using Feather`. I created a dataframe and saved it
with `Feather.write("/dir/dataframe.feather", df)` and it worked. I
can even open it back with `df =
Feather.read("/dir/dataframe.feather")` and get: `julia> nrow(df)
128544`.
The problem is with R. I am using Rstudio to test each step.
I installed the package, imported it with `library(feather)` and used it as:
```
df = read_feather("/dir/dataframe.feather")
```
and then Rstudio simply crashes.
any idea why?
Thank you

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Error exporting dataframe from Julia to R with Feather

Bert Gunter-2
You may have to contact RStudio about this. RStudio is a separate IDE
independent of R -- i.e. developed and maintained by a separate
organization from the R project.

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Thu, Aug 8, 2019 at 7:08 AM Luigi Marongiu <[hidden email]>
wrote:

> Hello,
>
> since I am encountering a lot of problems exporting dataframes from
> julia to R (there is always something wrong with the formatting,
> probably a missing quote) so I am trying to use Feather to do the job.
>
> I have installed Feather in Julia with `pkg.add("Feather")` and
> imported it with `using Feather`. I created a dataframe and saved it
> with `Feather.write("/dir/dataframe.feather", df)` and it worked. I
> can even open it back with `df =
> Feather.read("/dir/dataframe.feather")` and get: `julia> nrow(df)
> 128544`.
> The problem is with R. I am using Rstudio to test each step.
> I installed the package, imported it with `library(feather)` and used it
> as:
> ```
> df = read_feather("/dir/dataframe.feather")
> ```
> and then Rstudio simply crashes.
> any idea why?
> Thank you
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Error exporting dataframe from Julia to R with Feather

Peter Dalgaard-2
Alternatively, try running your example from plain R (in a terminal, R.app, or Rgui, depending on your platform), and see if the problem occurs without RStudio in the equation. If it does, then the feather package probably owns the problem.

-pd

> On 8 Aug 2019, at 16:34 , Bert Gunter <[hidden email]> wrote:
>
> You may have to contact RStudio about this. RStudio is a separate IDE
> independent of R -- i.e. developed and maintained by a separate
> organization from the R project.
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along and
> sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Thu, Aug 8, 2019 at 7:08 AM Luigi Marongiu <[hidden email]>
> wrote:
>
>> Hello,
>>
>> since I am encountering a lot of problems exporting dataframes from
>> julia to R (there is always something wrong with the formatting,
>> probably a missing quote) so I am trying to use Feather to do the job.
>>
>> I have installed Feather in Julia with `pkg.add("Feather")` and
>> imported it with `using Feather`. I created a dataframe and saved it
>> with `Feather.write("/dir/dataframe.feather", df)` and it worked. I
>> can even open it back with `df =
>> Feather.read("/dir/dataframe.feather")` and get: `julia> nrow(df)
>> 128544`.
>> The problem is with R. I am using Rstudio to test each step.
>> I installed the package, imported it with `library(feather)` and used it
>> as:
>> ```
>> df = read_feather("/dir/dataframe.feather")
>> ```
>> and then Rstudio simply crashes.
>> any idea why?
>> Thank you
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Error exporting dataframe from Julia to R with Feather

Bert Gunter-2
That is a better path, I agree.

However, I suspect that RStudio would still like to know about the issue,
even *if* feather/R is what crashes. They probably do not want RStudio to
crash even so.

-- Bert

On Thu, Aug 8, 2019 at 8:29 AM peter dalgaard <[hidden email]> wrote:

> Alternatively, try running your example from plain R (in a terminal,
> R.app, or Rgui, depending on your platform), and see if the problem occurs
> without RStudio in the equation. If it does, then the feather package
> probably owns the problem.
>
> -pd
>
> > On 8 Aug 2019, at 16:34 , Bert Gunter <[hidden email]> wrote:
> >
> > You may have to contact RStudio about this. RStudio is a separate IDE
> > independent of R -- i.e. developed and maintained by a separate
> > organization from the R project.
> >
> > Bert Gunter
> >
> > "The trouble with having an open mind is that people keep coming along
> and
> > sticking things into it."
> > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
> >
> >
> > On Thu, Aug 8, 2019 at 7:08 AM Luigi Marongiu <[hidden email]>
> > wrote:
> >
> >> Hello,
> >>
> >> since I am encountering a lot of problems exporting dataframes from
> >> julia to R (there is always something wrong with the formatting,
> >> probably a missing quote) so I am trying to use Feather to do the job.
> >>
> >> I have installed Feather in Julia with `pkg.add("Feather")` and
> >> imported it with `using Feather`. I created a dataframe and saved it
> >> with `Feather.write("/dir/dataframe.feather", df)` and it worked. I
> >> can even open it back with `df =
> >> Feather.read("/dir/dataframe.feather")` and get: `julia> nrow(df)
> >> 128544`.
> >> The problem is with R. I am using Rstudio to test each step.
> >> I installed the package, imported it with `library(feather)` and used it
> >> as:
> >> ```
> >> df = read_feather("/dir/dataframe.feather")
> >> ```
> >> and then Rstudio simply crashes.
> >> any idea why?
> >> Thank you
> >>
> >> ______________________________________________
> >> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide
> >> http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >>
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: [hidden email]  Priv: [hidden email]
>
>
>
>
>
>
>
>
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Error exporting dataframe from Julia to R with Feather

Jeff Newmiller
If R crashes, RStudio typically also crashes... so not necessarily news to them.

I will say that reproducible examples are nearly always necessary in cases like this to obtain meaningful answers from anyone, and the output of sessionInfo() is also usually needed.

On August 8, 2019 8:35:49 AM PDT, Bert Gunter <[hidden email]> wrote:

>That is a better path, I agree.
>
>However, I suspect that RStudio would still like to know about the
>issue,
>even *if* feather/R is what crashes. They probably do not want RStudio
>to
>crash even so.
>
>-- Bert
>
>On Thu, Aug 8, 2019 at 8:29 AM peter dalgaard <[hidden email]> wrote:
>
>> Alternatively, try running your example from plain R (in a terminal,
>> R.app, or Rgui, depending on your platform), and see if the problem
>occurs
>> without RStudio in the equation. If it does, then the feather package
>> probably owns the problem.
>>
>> -pd
>>
>> > On 8 Aug 2019, at 16:34 , Bert Gunter <[hidden email]>
>wrote:
>> >
>> > You may have to contact RStudio about this. RStudio is a separate
>IDE
>> > independent of R -- i.e. developed and maintained by a separate
>> > organization from the R project.
>> >
>> > Bert Gunter
>> >
>> > "The trouble with having an open mind is that people keep coming
>along
>> and
>> > sticking things into it."
>> > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>> >
>> >
>> > On Thu, Aug 8, 2019 at 7:08 AM Luigi Marongiu
><[hidden email]>
>> > wrote:
>> >
>> >> Hello,
>> >>
>> >> since I am encountering a lot of problems exporting dataframes
>from
>> >> julia to R (there is always something wrong with the formatting,
>> >> probably a missing quote) so I am trying to use Feather to do the
>job.
>> >>
>> >> I have installed Feather in Julia with `pkg.add("Feather")` and
>> >> imported it with `using Feather`. I created a dataframe and saved
>it
>> >> with `Feather.write("/dir/dataframe.feather", df)` and it worked.
>I
>> >> can even open it back with `df =
>> >> Feather.read("/dir/dataframe.feather")` and get: `julia> nrow(df)
>> >> 128544`.
>> >> The problem is with R. I am using Rstudio to test each step.
>> >> I installed the package, imported it with `library(feather)` and
>used it
>> >> as:
>> >> ```
>> >> df = read_feather("/dir/dataframe.feather")
>> >> ```
>> >> and then Rstudio simply crashes.
>> >> any idea why?
>> >> Thank you
>> >>
>> >> ______________________________________________
>> >> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> >> https://stat.ethz.ch/mailman/listinfo/r-help
>> >> PLEASE do read the posting guide
>> >> http://www.R-project.org/posting-guide.html
>> >> and provide commented, minimal, self-contained, reproducible code.
>> >>
>> >
>> >       [[alternative HTML version deleted]]
>> >
>> > ______________________________________________
>> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>>
>> --
>> Peter Dalgaard, Professor,
>> Center for Statistics, Copenhagen Business School
>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>> Phone: (+45)38153501
>> Office: A 4.23
>> Email: [hidden email]  Priv: [hidden email]
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
> [[alternative HTML version deleted]]
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Error exporting dataframe from Julia to R with Feather

Bert Gunter-2
Jeff:
Fair enough. As I have no data (no experience with crashes), I am happy to
defer to those who do.

Cheers,
Bert

On Thu, Aug 8, 2019 at 10:28 AM Jeff Newmiller <[hidden email]>
wrote:

> If R crashes, RStudio typically also crashes... so not necessarily news to
> them.
>
> I will say that reproducible examples are nearly always necessary in cases
> like this to obtain meaningful answers from anyone, and the output of
> sessionInfo() is also usually needed.
>
> On August 8, 2019 8:35:49 AM PDT, Bert Gunter <[hidden email]>
> wrote:
> >That is a better path, I agree.
> >
> >However, I suspect that RStudio would still like to know about the
> >issue,
> >even *if* feather/R is what crashes. They probably do not want RStudio
> >to
> >crash even so.
> >
> >-- Bert
> >
> >On Thu, Aug 8, 2019 at 8:29 AM peter dalgaard <[hidden email]> wrote:
> >
> >> Alternatively, try running your example from plain R (in a terminal,
> >> R.app, or Rgui, depending on your platform), and see if the problem
> >occurs
> >> without RStudio in the equation. If it does, then the feather package
> >> probably owns the problem.
> >>
> >> -pd
> >>
> >> > On 8 Aug 2019, at 16:34 , Bert Gunter <[hidden email]>
> >wrote:
> >> >
> >> > You may have to contact RStudio about this. RStudio is a separate
> >IDE
> >> > independent of R -- i.e. developed and maintained by a separate
> >> > organization from the R project.
> >> >
> >> > Bert Gunter
> >> >
> >> > "The trouble with having an open mind is that people keep coming
> >along
> >> and
> >> > sticking things into it."
> >> > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
> >> >
> >> >
> >> > On Thu, Aug 8, 2019 at 7:08 AM Luigi Marongiu
> ><[hidden email]>
> >> > wrote:
> >> >
> >> >> Hello,
> >> >>
> >> >> since I am encountering a lot of problems exporting dataframes
> >from
> >> >> julia to R (there is always something wrong with the formatting,
> >> >> probably a missing quote) so I am trying to use Feather to do the
> >job.
> >> >>
> >> >> I have installed Feather in Julia with `pkg.add("Feather")` and
> >> >> imported it with `using Feather`. I created a dataframe and saved
> >it
> >> >> with `Feather.write("/dir/dataframe.feather", df)` and it worked.
> >I
> >> >> can even open it back with `df =
> >> >> Feather.read("/dir/dataframe.feather")` and get: `julia> nrow(df)
> >> >> 128544`.
> >> >> The problem is with R. I am using Rstudio to test each step.
> >> >> I installed the package, imported it with `library(feather)` and
> >used it
> >> >> as:
> >> >> ```
> >> >> df = read_feather("/dir/dataframe.feather")
> >> >> ```
> >> >> and then Rstudio simply crashes.
> >> >> any idea why?
> >> >> Thank you
> >> >>
> >> >> ______________________________________________
> >> >> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> >> PLEASE do read the posting guide
> >> >> http://www.R-project.org/posting-guide.html
> >> >> and provide commented, minimal, self-contained, reproducible code.
> >> >>
> >> >
> >> >       [[alternative HTML version deleted]]
> >> >
> >> > ______________________________________________
> >> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >> > https://stat.ethz.ch/mailman/listinfo/r-help
> >> > PLEASE do read the posting guide
> >> http://www.R-project.org/posting-guide.html
> >> > and provide commented, minimal, self-contained, reproducible code.
> >>
> >> --
> >> Peter Dalgaard, Professor,
> >> Center for Statistics, Copenhagen Business School
> >> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> >> Phone: (+45)38153501
> >> Office: A 4.23
> >> Email: [hidden email]  Priv: [hidden email]
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >       [[alternative HTML version deleted]]
> >
> >______________________________________________
> >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> --
> Sent from my phone. Please excuse my brevity.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Error exporting dataframe from Julia to R with Feather

Luigi
In reply to this post by Peter Dalgaard-2
Hi, I ran from terminal:
```
$ Rscript try.R
[1] "library feather loaded"
[1] "open file"

 *** caught segfault ***
address (nil), cause 'memory not mapped'

Traceback:
 1: .Call(`_feather_openFeather`, path)
 2: openFeather(path)
 3: feather(path)
 4: read_feather("Hits_dedupl.feather")
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)
```

Perhaps is the problem with the original database rather than feather.
When I save the dataframe made with Julia with CSV.write and reload in
R with read.table I get the error
```
Warning message:
In scan(file = file, what = what, sep = sep, quote = quote, dec = dec,
: EOF within quoted string > str(hit) 'data.frame': 81365 obs. of 1
variable: $ Group.Sample.Start.Match_len.Read_len.Hit_len.Hit.ID: chr
"Normal\tA4\t6064657\t27\t101\t13669\tNC_038931.1 Thermoproteus tenax
virus 1 (TTV1) genome\tNC_038931"
"Normal\tA0\t6064658\t29\t101\t13669\tNC_038931.1 Thermoproteus tenax
virus 1 (TTV1) genome\tNC_038931"
"Normal\tA0\t6064659\t28\t101\t13669\tNC_038931.1 Thermoproteus tenax
virus 1 (TTV1) genome\tNC_038931"
"Normal\tA0\t6064660\t27\t101\t13669\tNC_038931.1 Thermoproteus tenax
virus 1 (TTV1) genome\tNC_038931" ...
```


On Thu, Aug 8, 2019 at 5:29 PM peter dalgaard <[hidden email]> wrote:

>
> Alternatively, try running your example from plain R (in a terminal, R.app, or Rgui, depending on your platform), and see if the problem occurs without RStudio in the equation. If it does, then the feather package probably owns the problem.
>
> -pd
>
> > On 8 Aug 2019, at 16:34 , Bert Gunter <[hidden email]> wrote:
> >
> > You may have to contact RStudio about this. RStudio is a separate IDE
> > independent of R -- i.e. developed and maintained by a separate
> > organization from the R project.
> >
> > Bert Gunter
> >
> > "The trouble with having an open mind is that people keep coming along and
> > sticking things into it."
> > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
> >
> >
> > On Thu, Aug 8, 2019 at 7:08 AM Luigi Marongiu <[hidden email]>
> > wrote:
> >
> >> Hello,
> >>
> >> since I am encountering a lot of problems exporting dataframes from
> >> julia to R (there is always something wrong with the formatting,
> >> probably a missing quote) so I am trying to use Feather to do the job.
> >>
> >> I have installed Feather in Julia with `pkg.add("Feather")` and
> >> imported it with `using Feather`. I created a dataframe and saved it
> >> with `Feather.write("/dir/dataframe.feather", df)` and it worked. I
> >> can even open it back with `df =
> >> Feather.read("/dir/dataframe.feather")` and get: `julia> nrow(df)
> >> 128544`.
> >> The problem is with R. I am using Rstudio to test each step.
> >> I installed the package, imported it with `library(feather)` and used it
> >> as:
> >> ```
> >> df = read_feather("/dir/dataframe.feather")
> >> ```
> >> and then Rstudio simply crashes.
> >> any idea why?
> >> Thank you
> >>
> >> ______________________________________________
> >> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide
> >> http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >>
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: [hidden email]  Priv: [hidden email]
>
>
>
>
>
>
>
>
>


--
Best regards,
Luigi

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.