Passing arguments between S4 methods fails within a function:bug? example with raster package.

[Joris FA Meys]

Dear all,

This problem came up initially while debugging a function, but it
seems to be a more general problem of R. I hope I'm wrong, but I can't
find another explanation. Let me illustrate with the raster package.

For an object "RasterLayer" (which inherits from Raster), there is a
method xyValues defined with the signature
(object="RasterLayer",xy="matrix"). There is also a method with
signature (object="Raster",xy="vector"). The only thing this method
does, is change xy into a matrix and then pass on to the next method
using callGeneric again. Arguments are passed.

Now this all works smoothly, as long as you stay in the global environment :
require(raster)

a <- raster()
a[] <- 1:ncell(a)

origin <- c(-80,50)
eff.dist <- 100000

unlist(xyValues(a,xy=origin,buffer=eff.dist))
[1] 14140 14141 14500 14501

Now let's make a very basic test function :

test <- function(x,orig.point){
    eff.distance <- 100000
    p <- unlist(xyValues(x,xy=orig.point,buffer=eff.distance))
    return(p)
}

This gives the following result :
> test(a,origin)
Error in .local(object, xy, ...) : object 'eff.distance' not found

huh? Apparently, eff.distance got lost somewhere in the parsetree (am
I saying this correctly?)

The funny thing is when we change origin to a matrix :
> origin <- matrix(origin,ncol=2)

> unlist(xyValues(a,xy=origin,buffer=eff.dist))
[1] 14140 14141 14500 14501

> test(a,origin)
[1] 14140 14141 14500 14501

It all works again! So something goes wrong with passing the arguments
from one method to another using callGeneric. Is this a bug in R or am
I missing something obvious?

The relevant code from the raster package :

setMethod("xyValues", signature(object='Raster', xy='vector'),
        function(object, xy, ...) {
                if (length(xy) == 2) {
                        callGeneric(object, matrix(xy, ncol=2), ...)
                } else {
                        stop('xy coordinates should be a two-column matrix or data.frame,
or a vector of two numbers.')
                }
        } )

setMethod("xyValues", signature(object='RasterLayer', xy='matrix'),
        function(object, xy, method='simple', buffer=NULL, fun=NULL, na.rm=TRUE) {

                if (dim(xy)[2] != 2) {
                        stop('xy has wrong dimensions; it should have 2 columns' )
                }

                if (! is.null(buffer)) {
                        return( .xyvBuf(object, xy, buffer, fun, na.rm=na.rm) )
                }

                if (method=='bilinear') {
                        return(.bilinearValue(object, xy))
                } else if (method=='simple') {
                        cells <- cellFromXY(object, xy)
                        return(.readCells(object, cells))
                } else {
                        stop('invalid method argument. Should be simple or bilinear.')
                }
        }
)



--
Joris Meys
Statistical consultant

Ghent University
Faculty of Bioscience Engineering
Department of Applied mathematics, biometrics and process control

tel : [phone]
[hidden email]
-------------------------------
Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Niels Richard Hansen-2]

Joris,

I looked at the problem. Here is a minimal example reproducing
the error

setGeneric("myplus",function(x,y,...) standardGeneric("myplus"))
setMethod("myplus",c(x="numeric",y="numeric"),
           function(x,y,z=0) x+y+z
           )
setMethod("myplus",c(x="numeric",y="list"),
           function(x,y,...) callGeneric(x,unlist(y),...)
           )

myplus(1,1)   ## 2; OK
myplus(1,1,1) ## 3; OK

myplus(1,list(1)) ## 2; OK
myplus(1,list(1),z=1) ## 3; OK

a.c1 <- 1
myplus(1,list(1),z=a.c1) ## 3; OK

test <- function(x,y) {
   a.c <- 1    ## Problem occurs when using '.' in variable name
   myplus(a,y,z=a.c)
}

test(1,list(1)) ## error
test(1,1)  ## 3; OK

It looks like a bug. I went through all environments from inside
the call of myplus("numeric","numeric") and a.c is nicely located
in parent.frame(6) with the value 1. However, when it has a '.' (dot)
in the variable name, the variable is not found. If it has a name
without a '.' (dot) there is no problem:

test <- function(x,y) {
   a <- 1
   myplus(a,y,z=a)
}

test(1,list(1)) ## 3; OK

- Niels

On 26/08/10 15.32, Joris Meys wrote:

> Dear all,
>
> This problem came up initially while debugging a function, but it
> seems to be a more general problem of R. I hope I'm wrong, but I can't
> find another explanation. Let me illustrate with the raster package.
>
> For an object "RasterLayer" (which inherits from Raster), there is a
> method xyValues defined with the signature
> (object="RasterLayer",xy="matrix"). There is also a method with
> signature (object="Raster",xy="vector"). The only thing this method
> does, is change xy into a matrix and then pass on to the next method
> using callGeneric again. Arguments are passed.
>
> Now this all works smoothly, as long as you stay in the global environment :
> require(raster)
>
> a<- raster()
> a[]<- 1:ncell(a)
>
> origin<- c(-80,50)
> eff.dist<- 100000
>
> unlist(xyValues(a,xy=origin,buffer=eff.dist))
> [1] 14140 14141 14500 14501
>
> Now let's make a very basic test function :
>
> test<- function(x,orig.point){
>      eff.distance<- 100000
>      p<- unlist(xyValues(x,xy=orig.point,buffer=eff.distance))
>      return(p)
> }
>
> This gives the following result :
>> test(a,origin)
> Error in .local(object, xy, ...) : object 'eff.distance' not found
>
> huh? Apparently, eff.distance got lost somewhere in the parsetree (am
> I saying this correctly?)
>
> The funny thing is when we change origin to a matrix :
>> origin<- matrix(origin,ncol=2)
>
>> unlist(xyValues(a,xy=origin,buffer=eff.dist))
> [1] 14140 14141 14500 14501
>
>> test(a,origin)
> [1] 14140 14141 14500 14501
>
> It all works again! So something goes wrong with passing the arguments
> from one method to another using callGeneric. Is this a bug in R or am
> I missing something obvious?
>
> The relevant code from the raster package :
>
> setMethod("xyValues", signature(object='Raster', xy='vector'),
> function(object, xy, ...) {
> if (length(xy) == 2) {
> callGeneric(object, matrix(xy, ncol=2), ...)
> } else {
> stop('xy coordinates should be a two-column matrix or data.frame,
> or a vector of two numbers.')
> }
> } )
>
> setMethod("xyValues", signature(object='RasterLayer', xy='matrix'),
> function(object, xy, method='simple', buffer=NULL, fun=NULL, na.rm=TRUE) {
>
> if (dim(xy)[2] != 2) {
> stop('xy has wrong dimensions; it should have 2 columns' )
> }
>
> if (! is.null(buffer)) {
> return( .xyvBuf(object, xy, buffer, fun, na.rm=na.rm) )
> }
>
> if (method=='bilinear') {
> return(.bilinearValue(object, xy))
> } else if (method=='simple') {
> cells<- cellFromXY(object, xy)
> return(.readCells(object, cells))
> } else {
> stop('invalid method argument. Should be simple or bilinear.')
> }
> }
> )
>
>
>

--
Niels Richard Hansen                     Web:   www.math.ku.dk/~richard
Associate Professor Email: [hidden email]
Department of Mathematical Sciences        [hidden email]
University of Copenhagen Skype: nielsrichardhansen.dk
Universitetsparken 5 Phone: [phone] (office)
2100 Copenhagen Ø        [phone] (mobile)
Denmark

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Martin Morgan]

  On 8/26/2010 8:43 AM, Niels Richard Hansen wrote:

> setGeneric("myplus",function(x,y,...) standardGeneric("myplus"))
> setMethod("myplus",c(x="numeric",y="numeric"),
>           function(x,y,z=0) x+y+z
>           )
> setMethod("myplus",c(x="numeric",y="list"),
>           function(x,y,...) callGeneric(x,unlist(y),...)
>           )
>
> myplus(1,1)   ## 2; OK
> myplus(1,1,1) ## 3; OK
>
> myplus(1,list(1)) ## 2; OK
> myplus(1,list(1),z=1) ## 3; OK
>
> a.c1 <- 1
> myplus(1,list(1),z=a.c1) ## 3; OK
>
> test <- function(x,y) {
>   a.c <- 1    ## Problem occurs when using '.' in variable name
>   myplus(a,y,z=a.c)
> }
>
> test(1,list(1)) ## error
> test(1,1)  ## 3; OK

I get an error in both instances, because in fact there is no variable
'a' defined (anywhere) in the session. I do get your results if I define
a global variable a <- 2

The reason for the original problem can be seen by looking at how the
methods are implemented (I added parentheses in the setMethod calls for
clarity)

 > showMethods(myplus, includeDef=TRUE)
Function: myplus (package .GlobalEnv)
x="numeric", y="list"
function (x, y, ...)
{
     callGeneric(x, unlist(y), ...)
}


x="numeric", y="numeric"
function (x, y, ...)
{
     .local <- function (x, y, z = 0)
     {
         x + y + z
     }
     .local(x, y, ...)
}

and in particular the c("numeric", "numeric") method has an extra
argument z, and so has a nested .local function. This is how the methods
package deals with method signatures that differ from the generic.

I think, roughly, that when c("numeric", "numeric") tries finally to
resolve it's argument 'x', it looks for a variable 'a' in the
environment two levels up (method --> generic) from where it is being
resolved. This is fine if the method has no .local environment (e.g., if
c("numeric", "numeric") where defined as function(x, y, ...) { x +y }).
I think that there is no general solution to this; the user could
callGeneric or evaluate symbols from arbitrarily nested functions within
either the calling or called methods themselves, and could arrive at a
particular method through callGeneric or other routes (e.g.,
callNextMethod) that are not consistent in terms of the implied frame of
evaluation. I could be mistaken.

A simple solution is to provided named arguments to callGeneric, e.g.,
callGeneric(x=x, y=unlist(y), ...).

These issues are likely to cause problems for eval / substitute
expressions like the one posted by Joris yesterday, where there are
implicit assumptions about the environment in which the experession is
being evaluated.

Martin

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[Niels Richard Hansen-2]

On 26/08/10 18.42, Martin Morgan wrote:

> On 8/26/2010 8:43 AM, Niels Richard Hansen wrote:
>> setGeneric("myplus",function(x,y,...) standardGeneric("myplus"))
>> setMethod("myplus",c(x="numeric",y="numeric"),
>> function(x,y,z=0) x+y+z
>> )
>> setMethod("myplus",c(x="numeric",y="list"),
>> function(x,y,...) callGeneric(x,unlist(y),...)
>> )
>>
>> myplus(1,1) ## 2; OK
>> myplus(1,1,1) ## 3; OK
>>
>> myplus(1,list(1)) ## 2; OK
>> myplus(1,list(1),z=1) ## 3; OK
>>
>> a.c1 <- 1
>> myplus(1,list(1),z=a.c1) ## 3; OK
>>
>> test <- function(x,y) {
>> a.c <- 1 ## Problem occurs when using '.' in variable name
>> myplus(a,y,z=a.c)
>> }
>>
>> test(1,list(1)) ## error
>> test(1,1) ## 3; OK
>
> I get an error in both instances, because in fact there is no variable 'a'
> defined (anywhere) in the session. I do get your results if I define a global
> variable a <- 2

Oops, right.

>
> The reason for the original problem can be seen by looking at how the methods
> are implemented (I added parentheses in the setMethod calls for clarity)
>
>  > showMethods(myplus, includeDef=TRUE)
> Function: myplus (package .GlobalEnv)
> x="numeric", y="list"
> function (x, y, ...)
> {
> callGeneric(x, unlist(y), ...)
> }
>
>
> x="numeric", y="numeric"
> function (x, y, ...)
> {
> .local <- function (x, y, z = 0)
> {
> x + y + z
> }
> .local(x, y, ...)
> }
>
> and in particular the c("numeric", "numeric") method has an extra argument z,
> and so has a nested .local function. This is how the methods package deals with
> method signatures that differ from the generic.
>
> I think, roughly, that when c("numeric", "numeric") tries finally to resolve
> it's argument 'x', it looks for a variable 'a' in the environment two levels up
> (method --> generic) from where it is being resolved.

I believe that is true. If 'a' is put into the environment two levels
up (using 'browser'), it is found.

> This is fine if the method
> has no .local environment (e.g., if c("numeric", "numeric") where defined as
> function(x, y, ...) { x +y }). I think that there is no general solution to
> this; the user could callGeneric or evaluate symbols from arbitrarily nested
> functions within either the calling or called methods themselves, and could
> arrive at a particular method through callGeneric or other routes (e.g.,
> callNextMethod) that are not consistent in terms of the implied frame of
> evaluation. I could be mistaken.

But as I understand it, the scoping rules solve such problems and
locate the variable in any environment at a higher level? Replacing
the callGeneric by a direct call of myplus removes the problem, it
seems, though 'a' will still be three levels up.

>
> A simple solution is to provided named arguments to callGeneric, e.g.,
> callGeneric(x=x, y=unlist(y), ...).

I don't see how that solves the problem.

- Niels

>
> These issues are likely to cause problems for eval / substitute expressions like
> the one posted by Joris yesterday, where there are implicit assumptions about
> the environment in which the experession is being evaluated.
>
> Martin
>

--
Niels Richard Hansen                     Web:   www.math.ku.dk/~richard
Associate Professor Email: [hidden email]
Department of Mathematical Sciences        [hidden email]
University of Copenhagen Skype: nielsrichardhansen.dk
Universitetsparken 5 Phone: [phone] (office)
2100 Copenhagen Ø        [phone] (mobile)
Denmark

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.