long vectors not supported yet

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

long vectors not supported yet

Loris Bennett-2
Hi,

I am running R 3.3.3 and getting the following error:

  Error in add_edges(res, edges = t(as.matrix(el[, 1:2])), attr = weight) :
    long vectors not supported yet: ../../src/include/Rinlinedfuns.h:138

when passing a 13 GB TransitionLayer object to shortestPath from the
package 'gdistance'.

The error, albeit in a different context, is discussed here:

  https://github.com/tidyverse/readr/issues/309
  https://github.com/tidyverse/readr/pull/433

However it is not clear to me quite what the fix was and what kind of
limit may still exist in subsequent R versions.

Is this a general problem/issue with R, or is this something more
package-specific which should be addressed on R-sig-Geo?

Cheers,

Loris

--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universit├Ąt Berlin         Email [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: long vectors not supported yet

Jeff Newmiller
This looks to me like a package development issue... which may be under discussion in R-sig-geo (search the archives), but more likely to be appropriate to discuss with the maintainer by email or through their development repository (R-forge, though it looks unused).
--
Sent from my phone. Please excuse my brevity.

On February 14, 2018 7:43:51 AM PST, Loris Bennett <[hidden email]> wrote:

>Hi,
>
>I am running R 3.3.3 and getting the following error:
>
>Error in add_edges(res, edges = t(as.matrix(el[, 1:2])), attr = weight)
>:
>   long vectors not supported yet: ../../src/include/Rinlinedfuns.h:138
>
>when passing a 13 GB TransitionLayer object to shortestPath from the
>package 'gdistance'.
>
>The error, albeit in a different context, is discussed here:
>
>  https://github.com/tidyverse/readr/issues/309
>  https://github.com/tidyverse/readr/pull/433
>
>However it is not clear to me quite what the fix was and what kind of
>limit may still exist in subsequent R versions.
>
>Is this a general problem/issue with R, or is this something more
>package-specific which should be addressed on R-sig-Geo?
>
>Cheers,
>
>Loris

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: long vectors not supported yet

Loris Bennett-2
Hi Jeff,

Jeff Newmiller <[hidden email]> writes:

>> Hi,
>>
>> I am running R 3.3.3 and getting the following error:
>>
>> Error in add_edges(res, edges = t(as.matrix(el[, 1:2])), attr = weight)
>> :
>>    long vectors not supported yet: ../../src/include/Rinlinedfuns.h:138
>>
>> when passing a 13 GB TransitionLayer object to shortestPath from the
>> package 'gdistance'.
>>
>> The error, albeit in a different context, is discussed here:
>>
>>   https://github.com/tidyverse/readr/issues/309
>>   https://github.com/tidyverse/readr/pull/433
>>
>> However it is not clear to me quite what the fix was and what kind of
>> limit may still exist in subsequent R versions.
>>
>> Is this a general problem/issue with R, or is this something more
>> package-specific which should be addressed on R-sig-Geo?

> This looks to me like a package development issue... which may be
> under discussion in R-sig-geo (search the archives), but more likely
> to be appropriate to discuss with the maintainer by email or through
> their development repository (R-forge, though it looks unused).

Thanks for the information.  I reported the issue to the developers of
'igraph', which is where 'add_edges' comes from, here

  https://github.com/igraph/rigraph/issues/255

The response there to the question of whether it is an R core or an
igraph issue was

  R core for sure, but probably both.

Is this indeed the case for R core?  To my naive eye, the error message
suggests more of a known limitation, rather than a bona fide bug.

Cheers,

Loris

--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universit├Ąt Berlin         Email [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: long vectors not supported yet

Jeff Newmiller
It does appear to be an error message from R [1] regarding the difference between the maximum length of a data vector (~2^59) and the maximum length of a matrix/array/data frame dimension (~2^31).

As to whether this is under consideration for change, I don't know but I would not want such information to affect how I work around the limitation today because it may come with bugs initially anyway.

[1] https://stackoverflow.com/questions/24335692/large-matrices-in-r-long-vectors-not-supported-yet
--
Sent from my phone. Please excuse my brevity.

On February 15, 2018 4:33:13 AM PST, Loris Bennett <[hidden email]> wrote:

>Hi Jeff,
>
>Jeff Newmiller <[hidden email]> writes:
>
>>> Hi,
>>>
>>> I am running R 3.3.3 and getting the following error:
>>>
>>> Error in add_edges(res, edges = t(as.matrix(el[, 1:2])), attr =
>weight)
>>> :
>>>    long vectors not supported yet:
>../../src/include/Rinlinedfuns.h:138
>>>
>>> when passing a 13 GB TransitionLayer object to shortestPath from the
>>> package 'gdistance'.
>>>
>>> The error, albeit in a different context, is discussed here:
>>>
>>>   https://github.com/tidyverse/readr/issues/309
>>>   https://github.com/tidyverse/readr/pull/433
>>>
>>> However it is not clear to me quite what the fix was and what kind
>of
>>> limit may still exist in subsequent R versions.
>>>
>>> Is this a general problem/issue with R, or is this something more
>>> package-specific which should be addressed on R-sig-Geo?
>
>> This looks to me like a package development issue... which may be
>> under discussion in R-sig-geo (search the archives), but more likely
>> to be appropriate to discuss with the maintainer by email or through
>> their development repository (R-forge, though it looks unused).
>
>Thanks for the information.  I reported the issue to the developers of
>'igraph', which is where 'add_edges' comes from, here
>
>  https://github.com/igraph/rigraph/issues/255
>
>The response there to the question of whether it is an R core or an
>igraph issue was
>
>  R core for sure, but probably both.
>
>Is this indeed the case for R core?  To my naive eye, the error message
>suggests more of a known limitation, rather than a bona fide bug.
>
>Cheers,
>
>Loris

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.