R Lapack – why a subset?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

R Lapack – why a subset?

baptiste.auguie
Hi,

Why doesn't R include a full Lapack but only a subset?

My cda package (now archived) relying on RcppArmadillo has broken multiple
times on CRAN over the past few years following updates in the underlying
Armadillo library, Every time it follows the same pattern: Armadillo adds a
function to solve a specialised linear system more efficiently, and the
corresponding Lapack routine is not included in the R Lapack subset used on
CRAN, causing breakage. The workaround so far has been an unhappy
compromise with Armadillo developers, adding a "crippled lapack" flag in
the RcppArmadillo configure script, that triggers the use of alternative
(suboptimal) routines and passes CRAN checks. Most RcppArmadillo-dependent
packages don't see the problem; mine seems to be the only one using complex
linear algebra. Two years ago a large number of Lapack routines were added*
to R, and this nicely solved the issue for a while. Unfortunately the same
problem resurfaced last year, with another missing Lapack routine, and my
package is now archived (though it works fine with an external Lapack).
More generally, this workaround is not satisfying for various reasons, and
so I want to ask: why does R ship only a subset of Lapack in the first
place?

Best regards,

baptiste

* Relevant commit at
https://github.com/wch/r-source/commit/98acd96f22eb795a933879c0d3f740e802855473

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R Lapack – why a subset?

R devel mailing list
I'm not too happy with this either, but I believe the reason is that there would be a significant extra maintenance burden consisting of things that is not being used by R itself. In particular, complex math routines are little used and have historically caused a number of issues with correct argument and return-value passing.

Notice that it is a bit of a can of worms involving matching up C and Fortran compilers, OS versions, routines being and not being present in current LAPACK, which may differ from the system-supplied one, etc.

-pd

> On 26 Mar 2018, at 07:49 , Baptiste Auguie <[hidden email]> wrote:
>
> Hi,
>
> Why doesn't R include a full Lapack but only a subset?
>
> My cda package (now archived) relying on RcppArmadillo has broken multiple
> times on CRAN over the past few years following updates in the underlying
> Armadillo library, Every time it follows the same pattern: Armadillo adds a
> function to solve a specialised linear system more efficiently, and the
> corresponding Lapack routine is not included in the R Lapack subset used on
> CRAN, causing breakage. The workaround so far has been an unhappy
> compromise with Armadillo developers, adding a "crippled lapack" flag in
> the RcppArmadillo configure script, that triggers the use of alternative
> (suboptimal) routines and passes CRAN checks. Most RcppArmadillo-dependent
> packages don't see the problem; mine seems to be the only one using complex
> linear algebra. Two years ago a large number of Lapack routines were added*
> to R, and this nicely solved the issue for a while. Unfortunately the same
> problem resurfaced last year, with another missing Lapack routine, and my
> package is now archived (though it works fine with an external Lapack).
> More generally, this workaround is not satisfying for various reasons, and
> so I want to ask: why does R ship only a subset of Lapack in the first
> place?
>
> Best regards,
>
> baptiste
>
> * Relevant commit at
> https://github.com/wch/r-source/commit/98acd96f22eb795a933879c0d3f740e802855473
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R Lapack – why a subset?

Keith O'Hara
Peter,

Is there not already a significant maintenance burden from cherry-picking routines? From my own experience (https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16482 <https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16482>) this is a painful process. And while users may not call these complex routines directly, R is often used as a front-end for libraries that do, so Rlapack-related linking errors are arising more and more.

Is the cost really so high as to preclude adding the remaining Lapack routines to Rlapack?

Best,
Keith

> On Mar 26, 2018, at 10:24 AM, Peter Dalgaard via R-devel <[hidden email]> wrote:
>
> I'm not too happy with this either, but I believe the reason is that there would be a significant extra maintenance burden consisting of things that is not being used by R itself. In particular, complex math routines are little used and have historically caused a number of issues with correct argument and return-value passing.
>
> Notice that it is a bit of a can of worms involving matching up C and Fortran compilers, OS versions, routines being and not being present in current LAPACK, which may differ from the system-supplied one, etc.
>
> -pd
>
>> On 26 Mar 2018, at 07:49 , Baptiste Auguie <[hidden email]> wrote:
>>
>> Hi,
>>
>> Why doesn't R include a full Lapack but only a subset?
>>
>> My cda package (now archived) relying on RcppArmadillo has broken multiple
>> times on CRAN over the past few years following updates in the underlying
>> Armadillo library, Every time it follows the same pattern: Armadillo adds a
>> function to solve a specialised linear system more efficiently, and the
>> corresponding Lapack routine is not included in the R Lapack subset used on
>> CRAN, causing breakage. The workaround so far has been an unhappy
>> compromise with Armadillo developers, adding a "crippled lapack" flag in
>> the RcppArmadillo configure script, that triggers the use of alternative
>> (suboptimal) routines and passes CRAN checks. Most RcppArmadillo-dependent
>> packages don't see the problem; mine seems to be the only one using complex
>> linear algebra. Two years ago a large number of Lapack routines were added*
>> to R, and this nicely solved the issue for a while. Unfortunately the same
>> problem resurfaced last year, with another missing Lapack routine, and my
>> package is now archived (though it works fine with an external Lapack).
>> More generally, this workaround is not satisfying for various reasons, and
>> so I want to ask: why does R ship only a subset of Lapack in the first
>> place?
>>
>> Best regards,
>>
>> baptiste
>>
>> * Relevant commit at
>> https://github.com/wch/r-source/commit/98acd96f22eb795a933879c0d3f740e802855473
>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: [hidden email]  Priv: [hidden email]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R Lapack – why a subset?

Juan Telleria Ruiz de Aguirre
> Is the cost really so high as to preclude adding the remaining Lapack routines to Rlapack?

Updating Lapack Libraries shall not break compatibility, and rather
provide bug fixes I guess.

> the reason is that there would be a significant extra maintenance burden consisting of things that is not being used by R itself.

I agree with Keith O'Hara when she says that *"R is often used as a
front-end for libraries that do, so Rlapack-related linking errors are
arising more and more"*.

Maybe we could propose R-Core some SVN patches by ourselves for
R-alpha that update LAPLACK libraries and see how it looks like... :S

> http://www.netlib.org/lapack/archives/

> https://github.com/wch/r-source/tree/trunk/src/modules/lapack

And in SVN:

> svn checkout https://svn.r-project.org/R/trunk/ R-devel
> svn update
> svn diff > patch.diff

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel