side-effect of calling functions via `::`

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

side-effect of calling functions via `::`

Simon Barthelmé
Dear list

I'm not sure whether this is a bug or an unavoidable consequence of the
way packages are loaded, but there can be surprising side effects of
calling a function via package::function. Here's an example using the
formula.tools package:

form <- a ~ b
as.character(form)
formula.tools::lhs(form)
as.character(form)

The first call to as.character returns:
[1] "~" "a" "b"
The second returns:
[1] "a ~ b"

The reason being that formula.tools has:
S3method(as.character,formula)
in its namespace, which quietly supersedes the default one. In my case
it led to a bug that was rather hard to track down because it looked
like non-deterministic behaviour.
Shouldn't there at least be a warning about such side effects, the way
library() tells you about masking?

Best

Simon Barthelme

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: side-effect of calling functions via `::`

Lionel Henry
A package should probably never register a S3 method unless it owns
either the generic or the class. Here `formula.tools` owns neither.
Instead of registering the method, it should export it like a regular
function. This way S3 dispatch is based on lexical scoping rather than
session-wide side effect.

Lionel

> On 1 sept. 2017, at 12:57, Simon Barthelmé <[hidden email]> wrote:
>
> Dear list
>
> I'm not sure whether this is a bug or an unavoidable consequence of the way packages are loaded, but there can be surprising side effects of calling a function via package::function. Here's an example using the formula.tools package:
>
> form <- a ~ b
> as.character(form)
> formula.tools::lhs(form)
> as.character(form)
>
> The first call to as.character returns:
> [1] "~" "a" "b"
> The second returns:
> [1] "a ~ b"
>
> The reason being that formula.tools has:
> S3method(as.character,formula)
> in its namespace, which quietly supersedes the default one. In my case it led to a bug that was rather hard to track down because it looked like non-deterministic behaviour.
> Shouldn't there at least be a warning about such side effects, the way library() tells you about masking?
>
> Best
>
> Simon Barthelme
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: side-effect of calling functions via `::`

Martin Maechler
In reply to this post by Simon Barthelmé
>>>>> Simon Barthelmé <[hidden email]>
>>>>>     on Fri, 1 Sep 2017 12:57:13 +0200 writes:

    > Dear list
    > I'm not sure whether this is a bug or an unavoidable consequence of the
    > way packages are loaded, but there can be surprising side effects of
    > calling a function via package::function. Here's an example using the
    > formula.tools package:

    > form <- a ~ b
    > as.character(form)
    > formula.tools::lhs(form)
    > as.character(form)

    > The first call to as.character returns:
    > [1] "~" "a" "b"
    > The second returns:
    > [1] "a ~ b"

    > The reason being that formula.tools has:
    > S3method(as.character,formula)
    > in its namespace, which quietly supersedes the default one.

Sure.

    > In my case it led to a bug that was rather hard to track
    > down because it looked like non-deterministic behaviour.

well, it shouldn't have been hard to track I think ... see below

    > Shouldn't there at least be a warning about such side effects, the way
    > library() tells you about masking?

The help page on  "::"  is pretty clear about the fact that the
namespace is loaded if necessary.

Personally I've got the impression that  <namespace>::<name>  is
much "overused" nowadays, notably in packages where I'd strongly
advocate using  importFrom() in NAMESPACE, so all this happens
at package load time, and then _not_ using `::` in the package
sources itself.

Many people seem to forget that every use of `::` is an R
function call and using it is ineffecient compared to just using
the already imported name.

Best,
Martin Maechler
ETH Zurich and R Core Team

    > Best
    > Simon Barthelme

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: side-effect of calling functions via `::`

Martin Maechler
In reply to this post by Lionel Henry
>>>>> Lionel Henry <[hidden email]>
>>>>>     on Fri, 1 Sep 2017 13:47:07 +0200 writes:

    > A package should probably never register a S3 method unless it owns
    > either the generic or the class.

I agree... (and typically it does "own" the class)

    > Here `formula.tools` owns neither.

i.e., it neither defines as.character() nor class "formula".

    > Instead of registering the method, it should export it like a regular
    > function. This way S3 dispatch is based on lexical scoping rather than
    > session-wide side effect.

I don't the 2nd sentence above is quite correct.  S3 method
registration should be done (in the case it should) and S3
dispatch is not just based on lexical scoping but also on S3
method registration.

    > Lionel

It is still the case that :: silently loads the namespace if
needed, and that "things may behave differently" after the use '::', because
loading a namespace does have an effect on the R session ...,
(and I still think  `::`  is much  "over used")

Martin



    >> On 1 sept. 2017, at 12:57, Simon Barthelmé <[hidden email]> wrote:
    >>
    >> Dear list
    >>
    >> I'm not sure whether this is a bug or an unavoidable consequence of the way packages are loaded, but there can be surprising side effects of calling a function via package::function. Here's an example using the formula.tools package:
    >>
    >> form <- a ~ b
    >> as.character(form)
    >> formula.tools::lhs(form)
    >> as.character(form)
    >>
    >> The first call to as.character returns:
    >> [1] "~" "a" "b"
    >> The second returns:
    >> [1] "a ~ b"
    >>
    >> The reason being that formula.tools has:
    >> S3method(as.character,formula)
    >> in its namespace, which quietly supersedes the default one. In my case it led to a bug that was rather hard to track down because it looked like non-deterministic behaviour.
    >> Shouldn't there at least be a warning about such side effects, the way library() tells you about masking?
    >>
    >> Best
    >>
    >> Simon Barthelme
    >>
    >> ______________________________________________
    >> [hidden email] mailing list
    >> https://stat.ethz.ch/mailman/listinfo/r-devel

    > ______________________________________________
    > [hidden email] mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: side-effect of calling functions via `::`

Simon Urbanek-3
Really, we have three levels of behavior related to dispatch: not loaded, loaded and attached. The loaded state is the most fragile - it does change some behavior (like the one below) but not others (when the package defines a new version of a generic). So it is true that the dispatch is the most problematic, in some sense you'd want it to be local to the package code when the package is not attached, but that's not supported in R as it is now.

Martin, re :: - I strongly disagree, semantically I find :: much cleaner than imports because you know exactly which function you call at all times and you don't pollute you package unnecessarily. It is unfortunate that the construct is inefficient in R, but that's an implementation problem, it shouldn't be because in fact it's immediately clear which symbol is meant. I believe the compiler should be able to fix that so in principle it shouldn't make a difference performance wise.

Cheers,
Simon


> On Sep 1, 2017, at 8:03 AM, Martin Maechler <[hidden email]> wrote:
>
>>>>>> Lionel Henry <[hidden email]>
>>>>>>    on Fri, 1 Sep 2017 13:47:07 +0200 writes:
>
>> A package should probably never register a S3 method unless it owns
>> either the generic or the class.
>
> I agree... (and typically it does "own" the class)
>
>> Here `formula.tools` owns neither.
>
> i.e., it neither defines as.character() nor class "formula".
>
>> Instead of registering the method, it should export it like a regular
>> function. This way S3 dispatch is based on lexical scoping rather than
>> session-wide side effect.
>
> I don't the 2nd sentence above is quite correct.  S3 method
> registration should be done (in the case it should) and S3
> dispatch is not just based on lexical scoping but also on S3
> method registration.
>
>> Lionel
>
> It is still the case that :: silently loads the namespace if
> needed, and that "things may behave differently" after the use '::', because
> loading a namespace does have an effect on the R session ...,
> (and I still think  `::`  is much  "over used")
>
> Martin
>
>
>
>>> On 1 sept. 2017, at 12:57, Simon Barthelmé <[hidden email]> wrote:
>>>
>>> Dear list
>>>
>>> I'm not sure whether this is a bug or an unavoidable consequence of the way packages are loaded, but there can be surprising side effects of calling a function via package::function. Here's an example using the formula.tools package:
>>>
>>> form <- a ~ b
>>> as.character(form)
>>> formula.tools::lhs(form)
>>> as.character(form)
>>>
>>> The first call to as.character returns:
>>> [1] "~" "a" "b"
>>> The second returns:
>>> [1] "a ~ b"
>>>
>>> The reason being that formula.tools has:
>>> S3method(as.character,formula)
>>> in its namespace, which quietly supersedes the default one. In my case it led to a bug that was rather hard to track down because it looked like non-deterministic behaviour.
>>> Shouldn't there at least be a warning about such side effects, the way library() tells you about masking?
>>>
>>> Best
>>>
>>> Simon Barthelme
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: side-effect of calling functions via `::`

Simon Urbanek
In reply to this post by Martin Maechler
Really, we have three levels of behavior related to dispatch: not loaded, loaded and attached. The loaded state is the most fragile - it does change some behavior (like the one below) but not others (when the package defines a new version of a generic). So it is true that the dispatch is the most problematic, in some sense you'd want it to be local to the package code when the package is not attached, but that's not supported in R as it is now.

Martin, re :: - I strongly disagree, semantically I find :: much cleaner than imports because you know exactly which function you call at all times and you don't pollute you package unnecessarily. It is unfortunate that the construct is inefficient in R, but that's an implementation problem, it shouldn't be because in fact it's immediately clear which symbol is meant. I believe the compiler should be able to fix that so in principle it shouldn't make a difference performance wise.

Cheers,
Simon


> On Sep 1, 2017, at 8:03 AM, Martin Maechler <[hidden email]> wrote:
>
>>>>>> Lionel Henry <[hidden email]>
>>>>>>   on Fri, 1 Sep 2017 13:47:07 +0200 writes:
>
>> A package should probably never register a S3 method unless it owns
>> either the generic or the class.
>
> I agree... (and typically it does "own" the class)
>
>> Here `formula.tools` owns neither.
>
> i.e., it neither defines as.character() nor class "formula".
>
>> Instead of registering the method, it should export it like a regular
>> function. This way S3 dispatch is based on lexical scoping rather than
>> session-wide side effect.
>
> I don't the 2nd sentence above is quite correct.  S3 method
> registration should be done (in the case it should) and S3
> dispatch is not just based on lexical scoping but also on S3
> method registration.
>
>> Lionel
>
> It is still the case that :: silently loads the namespace if
> needed, and that "things may behave differently" after the use '::', because
> loading a namespace does have an effect on the R session ...,
> (and I still think  `::`  is much  "over used")
>
> Martin
>
>
>
>>> On 1 sept. 2017, at 12:57, Simon Barthelmé <[hidden email]> wrote:
>>>
>>> Dear list
>>>
>>> I'm not sure whether this is a bug or an unavoidable consequence of the way packages are loaded, but there can be surprising side effects of calling a function via package::function. Here's an example using the formula.tools package:
>>>
>>> form <- a ~ b
>>> as.character(form)
>>> formula.tools::lhs(form)
>>> as.character(form)
>>>
>>> The first call to as.character returns:
>>> [1] "~" "a" "b"
>>> The second returns:
>>> [1] "a ~ b"
>>>
>>> The reason being that formula.tools has:
>>> S3method(as.character,formula)
>>> in its namespace, which quietly supersedes the default one. In my case it led to a bug that was rather hard to track down because it looked like non-deterministic behaviour.
>>> Shouldn't there at least be a warning about such side effects, the way library() tells you about masking?
>>>
>>> Best
>>>
>>> Simon Barthelme
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: side-effect of calling functions via `::`

Lionel Henry
> in some sense you'd want it to be local to the package code when the
> package is not attached, but that's not supported in R as it is now.

Lexically scoped methods work well (e.g. all methods in the base package)
but they are discouraged by a WARNING in R CMD check:

```
Found the following apparent S3 methods exported but not registered:
  as.character.formula
See section ‘Registering S3 methods’ in the ‘Writing R Extensions’
manual.
```

I think that's too bad because they seem like a legitimate way of
providing encapsulated dispatch.

Lionel


> On 1 sept. 2017, at 14:49, Simon Urbanek <[hidden email]> wrote:
>
> Really, we have three levels of behavior related to dispatch: not loaded, loaded and attached. The loaded state is the most fragile - it does change some behavior (like the one below) but not others (when the package defines a new version of a generic). So it is true that the dispatch is the most problematic, in some sense you'd want it to be local to the package code when the package is not attached, but that's not supported in R as it is now.
>
> Martin, re :: - I strongly disagree, semantically I find :: much cleaner than imports because you know exactly which function you call at all times and you don't pollute you package unnecessarily. It is unfortunate that the construct is inefficient in R, but that's an implementation problem, it shouldn't be because in fact it's immediately clear which symbol is meant. I believe the compiler should be able to fix that so in principle it shouldn't make a difference performance wise.
>
> Cheers,
> Simon
>
>
>> On Sep 1, 2017, at 8:03 AM, Martin Maechler <[hidden email]> wrote:
>>
>>>>>>> Lionel Henry <[hidden email]>
>>>>>>>  on Fri, 1 Sep 2017 13:47:07 +0200 writes:
>>
>>> A package should probably never register a S3 method unless it owns
>>> either the generic or the class.
>>
>> I agree... (and typically it does "own" the class)
>>
>>> Here `formula.tools` owns neither.
>>
>> i.e., it neither defines as.character() nor class "formula".
>>
>>> Instead of registering the method, it should export it like a regular
>>> function. This way S3 dispatch is based on lexical scoping rather than
>>> session-wide side effect.
>>
>> I don't the 2nd sentence above is quite correct.  S3 method
>> registration should be done (in the case it should) and S3
>> dispatch is not just based on lexical scoping but also on S3
>> method registration.
>>
>>> Lionel
>>
>> It is still the case that :: silently loads the namespace if
>> needed, and that "things may behave differently" after the use '::', because
>> loading a namespace does have an effect on the R session ...,
>> (and I still think  `::`  is much  "over used")
>>
>> Martin
>>
>>
>>
>>>> On 1 sept. 2017, at 12:57, Simon Barthelmé <[hidden email]> wrote:
>>>>
>>>> Dear list
>>>>
>>>> I'm not sure whether this is a bug or an unavoidable consequence of the way packages are loaded, but there can be surprising side effects of calling a function via package::function. Here's an example using the formula.tools package:
>>>>
>>>> form <- a ~ b
>>>> as.character(form)
>>>> formula.tools::lhs(form)
>>>> as.character(form)
>>>>
>>>> The first call to as.character returns:
>>>> [1] "~" "a" "b"
>>>> The second returns:
>>>> [1] "a ~ b"
>>>>
>>>> The reason being that formula.tools has:
>>>> S3method(as.character,formula)
>>>> in its namespace, which quietly supersedes the default one. In my case it led to a bug that was rather hard to track down because it looked like non-deterministic behaviour.
>>>> Shouldn't there at least be a warning about such side effects, the way library() tells you about masking?
>>>>
>>>> Best
>>>>
>>>> Simon Barthelme
>>>>
>>>> ______________________________________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: side-effect of calling functions via `::`

S Ellison-2
In reply to this post by Martin Maechler


> -----Original Message-----
> From: R-devel [mailto:[hidden email]] On Behalf Of Martin Maechler
> ...
> >>>>> Lionel Henry <[hidden email]>
>     > A package should probably never register a S3 method unless it owns
>     > either the generic or the class.
>
> I agree... (and typically it does "own" the class)

If that is true and a good general guide, is it worth adding something to that effect to 1.5.2 of "Writing R extensions"?
At present, nothing in 1.5.2 requires or recommends that a package using S3method owns either class or generic.


S Ellison


*******************************************************************
This email and any attachments are confidential. Any use...{{dropped:8}}

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel