'do.call' appears to show inconsistent behavior for arguments of format "pkg::fun"

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

'do.call' appears to show inconsistent behavior for arguments of format "pkg::fun"

Schlaepfer, Daniel
Hi all,

I believe that 'do.call' shows inconsistent/undocumented behavior when its 'what' argument is of the format "pkg::fun". The documentation (?do.call) suggests that
    what: either a function or a non-empty character string naming the function to be called.

Thus, I expected that all four of below formats of the value for the 'what' argument would work, here I use 'utils::person' as an example:
Three formats work:
> do.call(person, list(given = "Jane", family = "Doe"))
> do.call("person", list(given = "Jane", family = "Doe"))
> do.call(utils::person, list(given = "Jane", family = "Doe"))

but the format "pkg::fun" doesn't work as I expected:
> do.call("utils::person", list(given = "Jane", family = "Doe"))
  Error in `utils::person`(given = "Jane", family = "Doe") :
    could not find function "utils::person"

This seemingly inconsistent behavior of 'do.call' is also exposed when working in parallel and the message doesn't make it easy to see that this error derives actually from 'do.call'.

> library(parallel)
> cl <- makePSOCKcluster(2)

These work
> clusterCall(cl, person, list(given = "Jane", family = "Doe"))
> clusterCall(cl, "person", list(given = "Jane", family = "Doe"))
> clusterCall(cl, utils::person, list(given = "Jane", family = "Doe"))

This doesn't work
> clusterCall(cl, "utils::person", list(given = "Jane", family = "Doe"))
  Error in checkForRemoteErrors(lapply(cl, recvResult)) :
    2 nodes produced errors; first error: could not find function "utils::person"

This is again not obvious from the documentation (?clusterCall): fun, FUN: function or character string naming a function.

This behavior of 'clusterCall' is because the function 'makePSOCKcluster' calls newPSOCKnode() which calls .slaveRSOCK(), which calls slaveLoop(). This is then waiting and on receving an appropriate message will call 'do.call':
>  value <- tryCatch(do.call(msg$data$fun, msg$data$args, quote = TRUE), error = handler)

Thus, if 'msg$data$fun' (as received from recvData() which was sent via sendData(), via postNote() from function sendCall() wrapped in clusterCall()) is of the type "pkg::fun", then this will fail as above and work otherwise.

## Temporary work-around: re-define function in local namespace and export to workers
> temp_fun <- function(...) utils::person(...)
> clusterExport(cl, "temp_fun")
> clusterCall(cl, "temp_fun", list(given = "Jane", family = "Doe"))

Clean up
> stopCluster(cl)


## Motivation: I learned about this behavior of 'do.call' when attempting to write code such as
> Rmpi::mpi.bcast.cmd(mypackage2::fun, ...)

in my package1. The function Rmpi::mpi.bcast.cmd() calls eventually something along the lines of

> scmd <- scmd <- substitute(cmd)
> arg <- list(...)
> scmd.arg <-serialize(list(scmd=scmd, arg=arg), NULL)
> if (length(scmd.arg$args) > 0)
>    do.call(as.character(scmd.arg$$scmd), scmd.arg$args, envir = .GlobalEnv)

and thus expresses this same inconsistent behavior of do.call. I cannot avoid calling with pkg::fun because a package should not attach another package and change the search path of the user.



## My installation
> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-apple-darwin16.6.0 (64-bit)
Running under: macOS Sierra 10.12.5

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /opt/local/Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.4.1



I would appreciate any help with identifying whether this is indeed inconsistent/undocumented behavior of 'do.call' or whether I am simply missing the point and how to deal with the situation.

Thank you,
Daniel Schlaepfer

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'do.call' appears to show inconsistent behavior for arguments of format "pkg::fun"

Duncan Murdoch-2
On 10/07/2017 3:37 PM, Schlaepfer, Daniel wrote:
> Hi all,
>
> I believe that 'do.call' shows inconsistent/undocumented behavior when its 'what' argument is of the format "pkg::fun". The documentation (?do.call) suggests that
>     what: either a function or a non-empty character string naming the function to be called.

No, it is working just as documented.

>
> Thus, I expected that all four of below formats of the value for the 'what' argument would work, here I use 'utils::person' as an example:
> Three formats work:
>> do.call(person, list(given = "Jane", family = "Doe"))
>> do.call("person", list(given = "Jane", family = "Doe"))
>> do.call(utils::person, list(given = "Jane", family = "Doe"))
>
> but the format "pkg::fun" doesn't work as I expected:
>> do.call("utils::person", list(given = "Jane", family = "Doe"))
>   Error in `utils::person`(given = "Jane", family = "Doe") :
>     could not find function "utils::person"

person and utils::person are both expressions that give a function as
the value.  "person" and "utils::person" are both expressions that give
a character string, but "utils::person" is not the name of a function.

It's similar to this:

as.numeric(1)
as.numeric("1")
as.numeric(0+1)
as.numeric("0+1")

The 4th expression is not something that as.numeric knows how to coerce
to a number.  The rest are expressions that evaluate to 1 or "1", and
as.numeric() knows how to deal with both of those.


Duncan Murdoch

>
> This seemingly inconsistent behavior of 'do.call' is also exposed when working in parallel and the message doesn't make it easy to see that this error derives actually from 'do.call'.
>
>> library(parallel)
>> cl <- makePSOCKcluster(2)
>
> These work
>> clusterCall(cl, person, list(given = "Jane", family = "Doe"))
>> clusterCall(cl, "person", list(given = "Jane", family = "Doe"))
>> clusterCall(cl, utils::person, list(given = "Jane", family = "Doe"))
>
> This doesn't work
>> clusterCall(cl, "utils::person", list(given = "Jane", family = "Doe"))
>   Error in checkForRemoteErrors(lapply(cl, recvResult)) :
>     2 nodes produced errors; first error: could not find function "utils::person"
>
> This is again not obvious from the documentation (?clusterCall): fun, FUN: function or character string naming a function.
>
> This behavior of 'clusterCall' is because the function 'makePSOCKcluster' calls newPSOCKnode() which calls .slaveRSOCK(), which calls slaveLoop(). This is then waiting and on receving an appropriate message will call 'do.call':
>>  value <- tryCatch(do.call(msg$data$fun, msg$data$args, quote = TRUE), error = handler)
>
> Thus, if 'msg$data$fun' (as received from recvData() which was sent via sendData(), via postNote() from function sendCall() wrapped in clusterCall()) is of the type "pkg::fun", then this will fail as above and work otherwise.
>
> ## Temporary work-around: re-define function in local namespace and export to workers
>> temp_fun <- function(...) utils::person(...)
>> clusterExport(cl, "temp_fun")
>> clusterCall(cl, "temp_fun", list(given = "Jane", family = "Doe"))
>
> Clean up
>> stopCluster(cl)
>
>
> ## Motivation: I learned about this behavior of 'do.call' when attempting to write code such as
>> Rmpi::mpi.bcast.cmd(mypackage2::fun, ...)
>
> in my package1. The function Rmpi::mpi.bcast.cmd() calls eventually something along the lines of
>
>> scmd <- scmd <- substitute(cmd)
>> arg <- list(...)
>> scmd.arg <-serialize(list(scmd=scmd, arg=arg), NULL)
>> if (length(scmd.arg$args) > 0)
>>    do.call(as.character(scmd.arg$$scmd), scmd.arg$args, envir = .GlobalEnv)
>
> and thus expresses this same inconsistent behavior of do.call. I cannot avoid calling with pkg::fun because a package should not attach another package and change the search path of the user.
>
>
>
> ## My installation
>> sessionInfo()
> R version 3.4.1 (2017-06-30)
> Platform: x86_64-apple-darwin16.6.0 (64-bit)
> Running under: macOS Sierra 10.12.5
>
> Matrix products: default
> BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
> LAPACK: /opt/local/Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
>
> locale:
> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> loaded via a namespace (and not attached):
> [1] compiler_3.4.1
>
>
>
> I would appreciate any help with identifying whether this is indeed inconsistent/undocumented behavior of 'do.call' or whether I am simply missing the point and how to deal with the situation.
>
> Thank you,
> Daniel Schlaepfer
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'do.call' appears to show inconsistent behavior for arguments of format "pkg::fun"

R devel mailing list
In reply to this post by Schlaepfer, Daniel
>The function Rmpi::mpi.bcast.cmd() calls eventually something along the
lines of
>
>> scmd <- scmd <- substitute(cmd)
>> arg <- list(...)
>> scmd.arg <-serialize(list(scmd=scmd, arg=arg), NULL)
>> if (length(scmd.arg$args) > 0)
>>    do.call(as.character(scmd.arg$scmd), scmd.arg$args, envir =
.GlobalEnv)

Perhaps you or the maintainer of the Rmpi package should investigate why
that as.character() is needed in the call to do.call().  I suspect it is
not.



Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Mon, Jul 10, 2017 at 12:37 PM, Schlaepfer, Daniel <
[hidden email]> wrote:

> Hi all,
>
> I believe that 'do.call' shows inconsistent/undocumented behavior when its
> 'what' argument is of the format "pkg::fun". The documentation (?do.call)
> suggests that
>     what: either a function or a non-empty character string naming the
> function to be called.
>
> Thus, I expected that all four of below formats of the value for the
> 'what' argument would work, here I use 'utils::person' as an example:
> Three formats work:
> > do.call(person, list(given = "Jane", family = "Doe"))
> > do.call("person", list(given = "Jane", family = "Doe"))
> > do.call(utils::person, list(given = "Jane", family = "Doe"))
>
> but the format "pkg::fun" doesn't work as I expected:
> > do.call("utils::person", list(given = "Jane", family = "Doe"))
>   Error in `utils::person`(given = "Jane", family = "Doe") :
>     could not find function "utils::person"
>
> This seemingly inconsistent behavior of 'do.call' is also exposed when
> working in parallel and the message doesn't make it easy to see that this
> error derives actually from 'do.call'.
>
> > library(parallel)
> > cl <- makePSOCKcluster(2)
>
> These work
> > clusterCall(cl, person, list(given = "Jane", family = "Doe"))
> > clusterCall(cl, "person", list(given = "Jane", family = "Doe"))
> > clusterCall(cl, utils::person, list(given = "Jane", family = "Doe"))
>
> This doesn't work
> > clusterCall(cl, "utils::person", list(given = "Jane", family = "Doe"))
>   Error in checkForRemoteErrors(lapply(cl, recvResult)) :
>     2 nodes produced errors; first error: could not find function
> "utils::person"
>
> This is again not obvious from the documentation (?clusterCall): fun, FUN:
> function or character string naming a function.
>
> This behavior of 'clusterCall' is because the function 'makePSOCKcluster'
> calls newPSOCKnode() which calls .slaveRSOCK(), which calls slaveLoop().
> This is then waiting and on receving an appropriate message will call
> 'do.call':
> >  value <- tryCatch(do.call(msg$data$fun, msg$data$args, quote = TRUE),
> error = handler)
>
> Thus, if 'msg$data$fun' (as received from recvData() which was sent via
> sendData(), via postNote() from function sendCall() wrapped in
> clusterCall()) is of the type "pkg::fun", then this will fail as above and
> work otherwise.
>
> ## Temporary work-around: re-define function in local namespace and export
> to workers
> > temp_fun <- function(...) utils::person(...)
> > clusterExport(cl, "temp_fun")
> > clusterCall(cl, "temp_fun", list(given = "Jane", family = "Doe"))
>
> Clean up
> > stopCluster(cl)
>
>
> ## Motivation: I learned about this behavior of 'do.call' when attempting
> to write code such as
> > Rmpi::mpi.bcast.cmd(mypackage2::fun, ...)
>
> in my package1. The function Rmpi::mpi.bcast.cmd() calls eventually
> something along the lines of
>
> > scmd <- scmd <- substitute(cmd)
> > arg <- list(...)
> > scmd.arg <-serialize(list(scmd=scmd, arg=arg), NULL)
> > if (length(scmd.arg$args) > 0)
> >    do.call(as.character(scmd.arg$$scmd), scmd.arg$args, envir =
> .GlobalEnv)
>
> and thus expresses this same inconsistent behavior of do.call. I cannot
> avoid calling with pkg::fun because a package should not attach another
> package and change the search path of the user.
>
>
>
> ## My installation
> > sessionInfo()
> R version 3.4.1 (2017-06-30)
> Platform: x86_64-apple-darwin16.6.0 (64-bit)
> Running under: macOS Sierra 10.12.5
>
> Matrix products: default
> BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/
> A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
> LAPACK: /opt/local/Library/Frameworks/R.framework/Versions/3.4/
> Resources/lib/libRlapack.dylib
>
> locale:
> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> loaded via a namespace (and not attached):
> [1] compiler_3.4.1
>
>
>
> I would appreciate any help with identifying whether this is indeed
> inconsistent/undocumented behavior of 'do.call' or whether I am simply
> missing the point and how to deal with the situation.
>
> Thank you,
> Daniel Schlaepfer
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Loading...