Rscript fails with some packages (for example, h5)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Rscript fails with some packages (for example, h5)

Sun Yijiang
Consider this script (with h5 installed):

$ cat test.R
library(h5)
name <- tempfile()
f <- h5file(name)
file.remove(name)

$ Rscript test.R
Error in initialize(value, ...) :
  cannot use object of class "character" in new():  class "H5File" does not
extend that class
Calls: h5file -> new -> initialize -> initialize
Execution halted

$ /usr/lib64/R/bin/R --slave --no-restore --file=test.R
[1] TRUE

$ R_DEFAULT_PACKAGES= Rscript test.R
[1] TRUE

$ R
> source('test.R')
>

$ R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats' R
> source('test.R')
Error in initialize(value, ...) :
  cannot use object of class "character" in new():  class "H5File" does not
extend that class
>

After looking into C source code, I found that Rscript by default fills
environment variable R_DEFAULT_PACKAGES with
"datasets,utils,grDevices,graphics,stats", and it somehow fails some
package like h5.

The problem here is, not setting R_DEFAULT_PACKAGES is equivalent to
setting it to a magic value, it's really confusing.  I suggest remove this
feature.

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Rscript fails with some packages (for example, h5)

Dirk Eddelbuettel

On 26 December 2017 at 15:24, Sun Yijiang wrote:
| After looking into C source code, I found that Rscript by default fills
| environment variable R_DEFAULT_PACKAGES with
| "datasets,utils,grDevices,graphics,stats", and it somehow fails some
| package like h5.
|
| The problem here is, not setting R_DEFAULT_PACKAGES is equivalent to
| setting it to a magic value, it's really confusing.  I suggest remove this
| feature.

The more confusing part is that "methods" is missing 'by design' (as loading
methods is marginally more expensive that other packages). Ie for your script

   edd@bud:/tmp$ cat h5ex.R
   library(methods)
   library(h5)
   name <- tempfile()
   f <- h5file(name)
   file.remove(name)
   edd@bud:/tmp$ Rscript h5ex.R
   [1] TRUE
   edd@bud:/tmp$

it all works if you just add `library(methods)` as seen in the first line.

For what it is worth, littler's r does not need that as it loads methods just
like R itself does avoiding the confusion:

   edd@bud:/tmp$ cat h5ex2.R
   library(h5)
   name <- tempfile()
   f <- h5file(name)
   file.remove(name)
   edd@bud:/tmp$ r h5ex2.R
   edd@bud:/tmp$

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Rscript fails with some packages (for example, h5)

Sun Yijiang
Hi Dirk,

Thanks for the solution.  Now I know the work-arounds, but still don't
quite get it. Why does R_DEFAULT_PACKAGES has anything to do with
library(methods)?  If library(h5) works, it should just work, not depend on
an environment variable.  Rscript is not consistent with R, that's my
confusion.

Steve

2017-12-26 20:46 GMT+08:00 Dirk Eddelbuettel <[hidden email]>:

>
> On 26 December 2017 at 15:24, Sun Yijiang wrote:
> | After looking into C source code, I found that Rscript by default fills
> | environment variable R_DEFAULT_PACKAGES with
> | "datasets,utils,grDevices,graphics,stats", and it somehow fails some
> | package like h5.
> |
> | The problem here is, not setting R_DEFAULT_PACKAGES is equivalent to
> | setting it to a magic value, it's really confusing.  I suggest remove
> this
> | feature.
>
> The more confusing part is that "methods" is missing 'by design' (as
> loading
> methods is marginally more expensive that other packages). Ie for your
> script
>
>    edd@bud:/tmp$ cat h5ex.R
>    library(methods)
>    library(h5)
>    name <- tempfile()
>    f <- h5file(name)
>    file.remove(name)
>    edd@bud:/tmp$ Rscript h5ex.R
>    [1] TRUE
>    edd@bud:/tmp$
>
> it all works if you just add `library(methods)` as seen in the first line.
>
> For what it is worth, littler's r does not need that as it loads methods
> just
> like R itself does avoiding the confusion:
>
>    edd@bud:/tmp$ cat h5ex2.R
>    library(h5)
>    name <- tempfile()
>    f <- h5file(name)
>    file.remove(name)
>    edd@bud:/tmp$ r h5ex2.R
>    edd@bud:/tmp$
>
> Dirk
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Rscript fails with some packages (for example, h5)

Dirk Eddelbuettel

On 26 December 2017 at 22:14, Sun Yijiang wrote:
| Thanks for the solution.  Now I know the work-arounds, but still don't
| quite get it. Why does R_DEFAULT_PACKAGES has anything to do with
| library(methods)?

Because it governs which packages are loaded by default.  And while R also
loads 'methods', Rscript does not. Source of endless confusion.

| If library(h5) works, it should just work, not depend on an environment variable.

Every package using S4 will fail under Rscript unless 'methods' explicitly.

| Rscript is not consistent with R, that's my confusion.

Indeed. And you are not the first person confused by it.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Rscript fails with some packages (for example, h5)

Duncan Murdoch-2
On 26/12/2017 9:40 AM, Dirk Eddelbuettel wrote:
>
> On 26 December 2017 at 22:14, Sun Yijiang wrote:
> | Thanks for the solution.  Now I know the work-arounds, but still don't
> | quite get it. Why does R_DEFAULT_PACKAGES has anything to do with
> | library(methods)?
>
> Because it governs which packages are loaded by default.  And while R also
> loads 'methods', Rscript does not. Source of endless confusion.

Mostly irrelevant correction of the jargon:  that setting controls which
packages are "attached" by default.  library(h5) would be enough to load
methods, because h5 imports things from methods.  But loading doesn't
put a package on the search list.  library(methods) both loads methods
(if it hasn't already been loaded), and attaches it.

>
> | If library(h5) works, it should just work, not depend on an environment variable.
>
> Every package using S4 will fail under Rscript unless 'methods' explicitly.

That's not quite true (or quite English, as per fortune(112)).  The
"gmp" package imports methods, and it works in Rscript.  What doesn't
work is to expect library(h5) or library(gmp) to cause methods functions
like show() to be available to the user.

There is a way to do that, but it is discouraged nowadays. My ancient
package "orientlib" depends on methods, and that forces methods to be
attached so show() works in Rscript.  ("Depends" is used in the
DESCRIPTION file sense.)  h5 doesn't "depend" on methods, it "imports"
methods.


>
> | Rscript is not consistent with R, that's my confusion.
>
> Indeed. And you are not the first person confused by it.

And that's still true, and less irrelevant than my corrections.

Duncan Murdoch

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Rscript fails with some packages (for example, h5)

Dirk Eddelbuettel

Duncan,

Very nice tutorial. However it does NOT take away from the fact that _very_
simple_ scripts (like the one posted by Sun at the beginning of this thread)
simply _fail_ in error under Rscript.

Whereas they don't under R or r.

The R environment ships an interpreter meant for command-line and scripting
use which fails on simple scripts that happen to use S4. But I am tired of
arguing for reversing this as I have gotten nowhere in all those years.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Rscript fails with some packages (for example, h5)

Sun Yijiang
Thanks for the details. I’m new to R, and I’m not blaming anything here,
just that I’m still not clear what good it makes to keep this inconsistency
between R and Rscript. To me (and probably to many others from Perl/Python
etc.), this is shockingly weird. I can live with that, and I also want to
know why.

Steve
Dirk Eddelbuettel <[hidden email]>于2017年12月27日 周三21:15写道:

>
> Duncan,
>
> Very nice tutorial. However it does NOT take away from the fact that _very_
> simple_ scripts (like the one posted by Sun at the beginning of this
> thread)
> simply _fail_ in error under Rscript.
>
> Whereas they don't under R or r.
>
> The R environment ships an interpreter meant for command-line and scripting
> use which fails on simple scripts that happen to use S4. But I am tired of
> arguing for reversing this as I have gotten nowhere in all those years.
>
> Dirk
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Rscript fails with some packages (for example, h5)

Duncan Murdoch-2
On 27/12/2017 9:11 AM, Sun Yijiang wrote:
> Thanks for the details. I’m new to R, and I’m not blaming anything here,
> just that I’m still not clear what good it makes to keep this inconsistency
> between R and Rscript. To me (and probably to many others from Perl/Python
> etc.), this is shockingly weird. I can live with that, and I also want to
> know why.

As Dirk said, the argument was that the methods package used to take a
long time to load, and many uses of Rscript shouldn't have to pay that
time cost.  I don't know whether that's still true or not.  Dirk and I
agree that the inconsistency between R and Rscript is undesirable, but
neither of us is in a position to change it, or thinks it's worth the
effort to argue for a change.

Duncan


>
> Steve
> Dirk Eddelbuettel <[hidden email]>于2017年12月27日 周三21:15写道:
>
>>
>> Duncan,
>>
>> Very nice tutorial. However it does NOT take away from the fact that _very_
>> simple_ scripts (like the one posted by Sun at the beginning of this
>> thread)
>> simply _fail_ in error under Rscript.
>>
>> Whereas they don't under R or r.
>>
>> The R environment ships an interpreter meant for command-line and scripting
>> use which fails on simple scripts that happen to use S4. But I am tired of
>> arguing for reversing this as I have gotten nowhere in all those years.
>>
>> Dirk
>>
>> --
>> http://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]
>>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Rscript failing with h5 reveals bugs in h5 (and 'R CMD check')

Martin Maechler
In reply to this post by Duncan Murdoch-2
>>>>> Duncan Murdoch <[hidden email]>
>>>>>     on Wed, 27 Dec 2017 06:13:12 -0500 writes:

    > On 26/12/2017 9:40 AM, Dirk Eddelbuettel wrote:
    >>
    >> On 26 December 2017 at 22:14, Sun Yijiang wrote: | Thanks
    >> for the solution.  Now I know the work-arounds, but still
    >> don't | quite get it. Why does R_DEFAULT_PACKAGES has
    >> anything to do with | library(methods)?
    >>
    >> Because it governs which packages are loaded by default.
    >> And while R also loads 'methods', Rscript does
    >> not. Source of endless confusion.

    > Mostly irrelevant correction of the jargon: that setting
    > controls which packages are "attached" by default.
    > library(h5) would be enough to load methods, because h5
    > imports things from methods.  But loading doesn't put a
    > package on the search list.  library(methods) both loads
    > methods (if it hasn't already been loaded), and attaches
    > it.

    >>
    >> | If library(h5) works, it should just work, not depend
    >> on an environment variable.
    >>
    >> Every package using S4 will fail under Rscript unless
    >> 'methods' explicitly.

    > That's not quite true (or quite English, as per
    > fortune(112)).  The "gmp" package imports methods, and
    > it works in Rscript.  What doesn't work is to expect
    > library(h5) or library(gmp) to cause methods functions
    > like show() to be available to the user.

But indeed, in this case Sun's  test.R  script did not use any
such user level functions, and it still did not work when
methods is not attached...
and indeed that's the case also with R if you run it without
loading methods e.g. by

  R_DEFAULT_PACKAGES=NULL R CMD BATCH test.R

shows the same error...

===> There is really a bug in  h5 :  It does not import enough
from methods or it would all work fine, even with Rscript !!

===> So we have something relevant to R-devel , actually at
least one bug in R's  checking :

Why did
    R   CMD check --as-cran h5

not see that h5 defines methods for initialize() but never
imports that from methods ?
and so "should not work" when methods is not attached....

--

After all, the fact that the default packages attached at the
beginning differ between R and Rscript  has contributed to
revealing a bug in both 'h5' and R's checking procedures.

Maybe we should keep Rscript's incompatibility therefore ;-)

Martin

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel