Inefficiency in df$col

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Inefficiency in df$col

Radford Neal
While doing some performance testing with the new version of pqR (see
pqR-project.org), I've encountered an extreme, and quite unnecessary,
inefficiency in the current R Core implementation of R, which I think
you might want to correct.

The inefficiency is in access to columns of a data frame, as in
expressions such as df$col[i], which I think are very common (the
alternatives of df[i,"col"] and df[["col"]][i] are, I think, less
common).

Here is the setup for an example showing the issue:

  L <- list (abc=1:9, xyz=11:19)
  Lc <- L; class(Lc) <- "glub"
  df <- data.frame(L)

And here are some times for R-3.5.2 (r-devel of 2019-02-01 is much
the same):

  > system.time (for (i in 1:1000000) r <- L$xyz)
     user  system elapsed
    0.086   0.004   0.089
  > system.time (for (i in 1:1000000) r <- Lc$xyz)
     user  system elapsed
    0.494   0.000   0.495
  > system.time (for (i in 1:1000000) r <- df$xyz)
     user  system elapsed
    3.425   0.000   3.426

So accessing a column of a data frame is 38 times slower than
accessing a list element (which is what happens in the underlying
implementation of a data frame), and 7 times slower than accessing an
element of a list with a class attribute (for which it's necessary to
check whether there is a $.glub method, which there isn't here).

For comparison, here are the times for pqR-2019-01-25:

  > system.time (for (i in 1:1000000) r <- L$xyz)
     user  system elapsed
    0.057   0.000   0.058
  > system.time (for (i in 1:1000000) r <- Lc$xyz)
     user  system elapsed
    0.251   0.000   0.251
  > system.time (for (i in 1:1000000) r <- df$xyz)
     user  system elapsed
    0.247   0.000   0.247

So when accessing df$xyz, R-3.5.2 is 14 times slower than pqR-2019-01-25.
(For a partial match, like df$xy, R-3.5.2 is 34 times slower.)

I wasn't surprised that pqR was faster, but I didn't expect this big a
difference.  Then I remembered having seen a NEWS item from R-3.1.0:

  * Partial matching when using the $ operator _on data frames_ now
    throws a warning and may become defunct in the future. If partial
    matching is intended, replace foo$bar by foo[["bar", exact =
    FALSE]].

and having looked at the code then:

  `$.data.frame` <- function(x,name) {
    a <- x[[name]]
    if (!is.null(a)) return(a)
 
    a <- x[[name, exact=FALSE]]
    if (!is.null(a)) warning("Name partially matched in data frame")
    return(a)
  }

I recall thinking at the time that this involved a pretty big
performance hit, compared to letting the primitive $ operator do it,
just to produce a warning.  But it wasn't until now that I noticed
this NEWS in R-3.1.1:

  * The warning when using partial matching with the $ operator on
    data frames is now only given when
    options("warnPartialMatchDollar") is TRUE.

for which the code was changed to:

  `$.data.frame` <- function(x,name) {
    a <- x[[name]]
    if (!is.null(a)) return(a)
 
    a <- x[[name, exact=FALSE]]
    if (!is.null(a) && getOption("warnPartialMatchDollar", default=FALSE)) {
          names <- names(x)
          warning(gettextf("Partial match of '%s' to '%s' in data frame",
                                     name, names[pmatch(name, names)]))
    }
    return(a)
  }

One can see the effect now when warnPartialMatchDollar is enabled:

  > options(warnPartialMatchDollar=TRUE)
  > Lc$xy
  [1] 11 12 13 14 15 16 17 18 19
  Warning message:
  In Lc$xy : partial match of 'xy' to 'xyz'
  > df$xy
  [1] 11 12 13 14 15 16 17 18 19
  Warning message:
  In `$.data.frame`(df, xy) : Partial match of 'xy' to 'xyz' in data frame

So the only thing that slowing down acesses like df$xyz by a factor of
seven achieves now is to add the words "in data frame" to the warning
message (while making the earlier part of the message less intelligible).

I think you might want to just delete the definition of $.data.frame,
reverting to the situation before R-3.1.0.

   Radford Neal

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Inefficiency in df$col

Duncan Murdoch-2
On 03/02/2019 12:04 p.m., Radford Neal wrote:

> While doing some performance testing with the new version of pqR (see
> pqR-project.org), I've encountered an extreme, and quite unnecessary,
> inefficiency in the current R Core implementation of R, which I think
> you might want to correct.
>
> The inefficiency is in access to columns of a data frame, as in
> expressions such as df$col[i], which I think are very common (the
> alternatives of df[i,"col"] and df[["col"]][i] are, I think, less
> common).
>
> Here is the setup for an example showing the issue:
>
>    L <- list (abc=1:9, xyz=11:19)
>    Lc <- L; class(Lc) <- "glub"
>    df <- data.frame(L)
>
> And here are some times for R-3.5.2 (r-devel of 2019-02-01 is much
> the same):
>
>    > system.time (for (i in 1:1000000) r <- L$xyz)
>       user  system elapsed
>      0.086   0.004   0.089
>    > system.time (for (i in 1:1000000) r <- Lc$xyz)
>       user  system elapsed
>      0.494   0.000   0.495
>    > system.time (for (i in 1:1000000) r <- df$xyz)
>       user  system elapsed
>      3.425   0.000   3.426
>
> So accessing a column of a data frame is 38 times slower than
> accessing a list element (which is what happens in the underlying
> implementation of a data frame), and 7 times slower than accessing an
> element of a list with a class attribute (for which it's necessary to
> check whether there is a $.glub method, which there isn't here).
>
> For comparison, here are the times for pqR-2019-01-25:
>
>    > system.time (for (i in 1:1000000) r <- L$xyz)
>       user  system elapsed
>      0.057   0.000   0.058
>    > system.time (for (i in 1:1000000) r <- Lc$xyz)
>       user  system elapsed
>      0.251   0.000   0.251
>    > system.time (for (i in 1:1000000) r <- df$xyz)
>       user  system elapsed
>      0.247   0.000   0.247
>
> So when accessing df$xyz, R-3.5.2 is 14 times slower than pqR-2019-01-25.
> (For a partial match, like df$xy, R-3.5.2 is 34 times slower.)
>
> I wasn't surprised that pqR was faster, but I didn't expect this big a
> difference.  Then I remembered having seen a NEWS item from R-3.1.0:
>
>    * Partial matching when using the $ operator _on data frames_ now
>      throws a warning and may become defunct in the future. If partial
>      matching is intended, replace foo$bar by foo[["bar", exact =
>      FALSE]].
>
> and having looked at the code then:
>
>    `$.data.frame` <- function(x,name) {
>      a <- x[[name]]
>      if (!is.null(a)) return(a)
>    
>      a <- x[[name, exact=FALSE]]
>      if (!is.null(a)) warning("Name partially matched in data frame")
>      return(a)
>    }
>
> I recall thinking at the time that this involved a pretty big
> performance hit, compared to letting the primitive $ operator do it,
> just to produce a warning.  But it wasn't until now that I noticed
> this NEWS in R-3.1.1:
>
>    * The warning when using partial matching with the $ operator on
>      data frames is now only given when
>      options("warnPartialMatchDollar") is TRUE.
>
> for which the code was changed to:
>
>    `$.data.frame` <- function(x,name) {
>      a <- x[[name]]
>      if (!is.null(a)) return(a)
>    
>      a <- x[[name, exact=FALSE]]
>      if (!is.null(a) && getOption("warnPartialMatchDollar", default=FALSE)) {
>            names <- names(x)
>            warning(gettextf("Partial match of '%s' to '%s' in data frame",
>                                       name, names[pmatch(name, names)]))
>      }
>      return(a)
>    }
>
> One can see the effect now when warnPartialMatchDollar is enabled:
>
>    > options(warnPartialMatchDollar=TRUE)
>    > Lc$xy
>    [1] 11 12 13 14 15 16 17 18 19
>    Warning message:
>    In Lc$xy : partial match of 'xy' to 'xyz'
>    > df$xy
>    [1] 11 12 13 14 15 16 17 18 19
>    Warning message:
>    In `$.data.frame`(df, xy) : Partial match of 'xy' to 'xyz' in data frame
>
> So the only thing that slowing down acesses like df$xyz by a factor of
> seven achieves now is to add the words "in data frame" to the warning
> message (while making the earlier part of the message less intelligible).
>
> I think you might want to just delete the definition of $.data.frame,
> reverting to the situation before R-3.1.0.

I imagine the cause is that the list version is done in C code rather
than R code (i.e. there's no R function `$.list`).  So an alternative
solution would be to also implement `$.data.frame` in the underlying C
code.  This won't be quite as fast (it needs that test for NULL), but
should be close in the full match case.

Duncan Murdoch

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Inefficiency in df$col

Radford Neal
In reply to this post by Radford Neal
> > I think you might want to just delete the definition of $.data.frame,
> > reverting to the situation before R-3.1.0.
>
> I imagine the cause is that the list version is done in C code rather
> than R code (i.e. there's no R function `$.list`).  So an alternative
> solution would be to also implement `$.data.frame` in the underlying C
> code.  This won't be quite as fast (it needs that test for NULL), but
> should be close in the full match case.

I maybe wasn't completely clear.  The $ operator for data frames was
previously done in C - since it was done by the same primitive as for
lists.  In R-3.1.0, this was changed - producing a massive slowdown -
for the purpose of giving a warning on partial matches even if the
user had not set the warnPartialMatchDollar option to TRUE.  In
R-3.1.1, this was changed to not warn unless warnPartialMatchDollar was
TRUE which was the PREVIOUS behaviour.  In other words, this change
reverted the change made in R-3.1.0.  But instead of simply deleting
the definition of $.data.frame, R-3.1.1 added extra code to it, the
only effect of which is to slightly change the wording of the warning
message from what is produced for any other list, while still retaining
the massive slowdown.

There is no need for you to write $.data.frame in C.  You just need
to delete the version written in R.

   Radford Neal

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Inefficiency in df$col

Duncan Murdoch-2
On 04/02/2019 9:20 a.m., Radford Neal wrote:

>>> I think you might want to just delete the definition of $.data.frame,
>>> reverting to the situation before R-3.1.0.
>>
>> I imagine the cause is that the list version is done in C code rather
>> than R code (i.e. there's no R function `$.list`).  So an alternative
>> solution would be to also implement `$.data.frame` in the underlying C
>> code.  This won't be quite as fast (it needs that test for NULL), but
>> should be close in the full match case.
>
> I maybe wasn't completely clear.  The $ operator for data frames was
> previously done in C - since it was done by the same primitive as for
> lists.  In R-3.1.0, this was changed - producing a massive slowdown -
> for the purpose of giving a warning on partial matches even if the
> user had not set the warnPartialMatchDollar option to TRUE.  In
> R-3.1.1, this was changed to not warn unless warnPartialMatchDollar was
> TRUE which was the PREVIOUS behaviour.  In other words, this change
> reverted the change made in R-3.1.0.  But instead of simply deleting
> the definition of $.data.frame, R-3.1.1 added extra code to it, the
> only effect of which is to slightly change the wording of the warning
> message from what is produced for any other list, while still retaining
> the massive slowdown.
>
> There is no need for you to write $.data.frame in C.  You just need
> to delete the version written in R.

Sorry, I did misunderstand.  Thanks for the clarification.

But if the "You" in your last sentence meant me, it needs to be "They":
I am not a member of R Core and can't make any changes to the sources.

Duncan Murdoch

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Inefficiency in df$col

Peter Dalgaard-2
Does either of you have a patch against current R-devel?

I tried the obvious, but the build dies with

building package 'tools'
all.R is unchanged
../../../../library/tools/libs/x86_64/tools.so is unchanged
installing 'sysdata.rda'
Error in get(method, envir = home) : object '$.data.frame' not found
Error: unable to load R code in package 'tools'
Execution halted

...and I can't really be arsed to dig into tools to see exactly where it is hardcoding the existence of $.data.frame.

-pd

> On 4 Feb 2019, at 15:32 , Duncan Murdoch <[hidden email]> wrote:
>
> On 04/02/2019 9:20 a.m., Radford Neal wrote:
>>>> I think you might want to just delete the definition of $.data.frame,
>>>> reverting to the situation before R-3.1.0.
>>>
>>> I imagine the cause is that the list version is done in C code rather
>>> than R code (i.e. there's no R function `$.list`).  So an alternative
>>> solution would be to also implement `$.data.frame` in the underlying C
>>> code.  This won't be quite as fast (it needs that test for NULL), but
>>> should be close in the full match case.
>> I maybe wasn't completely clear.  The $ operator for data frames was
>> previously done in C - since it was done by the same primitive as for
>> lists.  In R-3.1.0, this was changed - producing a massive slowdown -
>> for the purpose of giving a warning on partial matches even if the
>> user had not set the warnPartialMatchDollar option to TRUE.  In
>> R-3.1.1, this was changed to not warn unless warnPartialMatchDollar was
>> TRUE which was the PREVIOUS behaviour.  In other words, this change
>> reverted the change made in R-3.1.0.  But instead of simply deleting
>> the definition of $.data.frame, R-3.1.1 added extra code to it, the
>> only effect of which is to slightly change the wording of the warning
>> message from what is produced for any other list, while still retaining
>> the massive slowdown.
>> There is no need for you to write $.data.frame in C.  You just need
>> to delete the version written in R.
>
> Sorry, I did misunderstand.  Thanks for the clarification.
>
> But if the "You" in your last sentence meant me, it needs to be "They": I am not a member of R Core and can't make any changes to the sources.
>
> Duncan Murdoch
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Inefficiency in df$col

Martin Maechler
>>>>> peter dalgaard
>>>>>     on Mon, 4 Feb 2019 16:48:12 +0100 writes:

    > Does either of you have a patch against current R-devel?
    > I tried the obvious, but the build dies with

    > building package 'tools'
    > all.R is unchanged
    > ../../../../library/tools/libs/x86_64/tools.so is unchanged
    > installing 'sysdata.rda'
    > Error in get(method, envir = home) : object '$.data.frame' not found
    > Error: unable to load R code in package 'tools'
    > Execution halted

    > ...and I can't really be arsed to dig into tools to see exactly where it is hardcoding the existence of $.data.frame.

    > -pd

Well, we two have been working "in parallel"...

I've just sent an e-mail to R-core about this:

It's really  file.size() which does need a 'data.frame' method for `$`
because it is basically a wrapper  file.info(..)$size  and
file.info(..) does return a data frame.

I've been suggesting the byte compiler's optimizations here to be
the problem ...

    >> On 4 Feb 2019, at 15:32 , Duncan Murdoch <[hidden email]> wrote:
    >>
    >> On 04/02/2019 9:20 a.m., Radford Neal wrote:
    >>>>> I think you might want to just delete the definition of $.data.frame,
    >>>>> reverting to the situation before R-3.1.0.
    >>>>
    >>>> I imagine the cause is that the list version is done in C code rather
    >>>> than R code (i.e. there's no R function `$.list`).  So an alternative
    >>>> solution would be to also implement `$.data.frame` in the underlying C
    >>>> code.  This won't be quite as fast (it needs that test for NULL), but
    >>>> should be close in the full match case.
    >>> I maybe wasn't completely clear.  The $ operator for data frames was
    >>> previously done in C - since it was done by the same primitive as for
    >>> lists.  In R-3.1.0, this was changed - producing a massive slowdown -
    >>> for the purpose of giving a warning on partial matches even if the
    >>> user had not set the warnPartialMatchDollar option to TRUE.  In
    >>> R-3.1.1, this was changed to not warn unless warnPartialMatchDollar was
    >>> TRUE which was the PREVIOUS behaviour.  In other words, this change
    >>> reverted the change made in R-3.1.0.  But instead of simply deleting
    >>> the definition of $.data.frame, R-3.1.1 added extra code to it, the
    >>> only effect of which is to slightly change the wording of the warning
    >>> message from what is produced for any other list, while still retaining
    >>> the massive slowdown.
    >>> There is no need for you to write $.data.frame in C.  You just need
    >>> to delete the version written in R.
    >>
    >> Sorry, I did misunderstand.  Thanks for the clarification.
    >>
    >> But if the "You" in your last sentence meant me, it needs to be "They": I am not a member of R Core and can't make any changes to the sources.
    >>
    >> Duncan Murdoch
    >>
    >> ______________________________________________
    >> [hidden email] mailing list
    >> https://stat.ethz.ch/mailman/listinfo/r-devel

    > --
    > Peter Dalgaard, Professor,
    > Center for Statistics, Copenhagen Business School
    > Solbjerg Plads 3, 2000 Frederiksberg, Denmark
    > Phone: (+45)38153501
    > Office: A 4.23
    > Email: [hidden email]  Priv: [hidden email]

    > ______________________________________________
    > [hidden email] mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Inefficiency in df$col

Duncan Murdoch-2
In reply to this post by Peter Dalgaard-2
On 04/02/2019 10:48 a.m., peter dalgaard wrote:

> Does either of you have a patch against current R-devel?
>
> I tried the obvious, but the build dies with
>
> building package 'tools'
> all.R is unchanged
> ../../../../library/tools/libs/x86_64/tools.so is unchanged
> installing 'sysdata.rda'
> Error in get(method, envir = home) : object '$.data.frame' not found
> Error: unable to load R code in package 'tools'
> Execution halted
>
> ...and I can't really be arsed to dig into tools to see exactly where it is hardcoding the existence of $.data.frame.

Since $.data.frame is in the base package, it is handled specially in
src/library/base/R/zzz.R.  I'll email you a patch if it compiles and
passes checks.

Duncan Murdoch

>
> -pd
>
>> On 4 Feb 2019, at 15:32 , Duncan Murdoch <[hidden email]> wrote:
>>
>> On 04/02/2019 9:20 a.m., Radford Neal wrote:
>>>>> I think you might want to just delete the definition of $.data.frame,
>>>>> reverting to the situation before R-3.1.0.
>>>>
>>>> I imagine the cause is that the list version is done in C code rather
>>>> than R code (i.e. there's no R function `$.list`).  So an alternative
>>>> solution would be to also implement `$.data.frame` in the underlying C
>>>> code.  This won't be quite as fast (it needs that test for NULL), but
>>>> should be close in the full match case.
>>> I maybe wasn't completely clear.  The $ operator for data frames was
>>> previously done in C - since it was done by the same primitive as for
>>> lists.  In R-3.1.0, this was changed - producing a massive slowdown -
>>> for the purpose of giving a warning on partial matches even if the
>>> user had not set the warnPartialMatchDollar option to TRUE.  In
>>> R-3.1.1, this was changed to not warn unless warnPartialMatchDollar was
>>> TRUE which was the PREVIOUS behaviour.  In other words, this change
>>> reverted the change made in R-3.1.0.  But instead of simply deleting
>>> the definition of $.data.frame, R-3.1.1 added extra code to it, the
>>> only effect of which is to slightly change the wording of the warning
>>> message from what is produced for any other list, while still retaining
>>> the massive slowdown.
>>> There is no need for you to write $.data.frame in C.  You just need
>>> to delete the version written in R.
>>
>> Sorry, I did misunderstand.  Thanks for the clarification.
>>
>> But if the "You" in your last sentence meant me, it needs to be "They": I am not a member of R Core and can't make any changes to the sources.
>>
>> Duncan Murdoch
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Inefficiency in df$col

Duncan Murdoch-2
In reply to this post by Martin Maechler
On 04/02/2019 11:34 a.m., Martin Maechler wrote:

>>>>>> peter dalgaard
>>>>>>      on Mon, 4 Feb 2019 16:48:12 +0100 writes:
>
>      > Does either of you have a patch against current R-devel?
>      > I tried the obvious, but the build dies with
>
>      > building package 'tools'
>      > all.R is unchanged
>      > ../../../../library/tools/libs/x86_64/tools.so is unchanged
>      > installing 'sysdata.rda'
>      > Error in get(method, envir = home) : object '$.data.frame' not found
>      > Error: unable to load R code in package 'tools'
>      > Execution halted
>
>      > ...and I can't really be arsed to dig into tools to see exactly where it is hardcoding the existence of $.data.frame.
>
>      > -pd
>
> Well, we two have been working "in parallel"...
>
> I've just sent an e-mail to R-core about this:
>
> It's really  file.size() which does need a 'data.frame' method for `$`
> because it is basically a wrapper  file.info(..)$size  and
> file.info(..) does return a data frame.
>
> I've been suggesting the byte compiler's optimizations here to be
> the problem ...

I think Radford's point is that there is no difference between the
behaviour of $ on a data.frame or a list (except for the wording of the
warning message), so the $.list method (which is fast) is sufficient.

Duncan Murdoch

>
>      >> On 4 Feb 2019, at 15:32 , Duncan Murdoch <[hidden email]> wrote:
>      >>
>      >> On 04/02/2019 9:20 a.m., Radford Neal wrote:
>      >>>>> I think you might want to just delete the definition of $.data.frame,
>      >>>>> reverting to the situation before R-3.1.0.
>      >>>>
>      >>>> I imagine the cause is that the list version is done in C code rather
>      >>>> than R code (i.e. there's no R function `$.list`).  So an alternative
>      >>>> solution would be to also implement `$.data.frame` in the underlying C
>      >>>> code.  This won't be quite as fast (it needs that test for NULL), but
>      >>>> should be close in the full match case.
>      >>> I maybe wasn't completely clear.  The $ operator for data frames was
>      >>> previously done in C - since it was done by the same primitive as for
>      >>> lists.  In R-3.1.0, this was changed - producing a massive slowdown -
>      >>> for the purpose of giving a warning on partial matches even if the
>      >>> user had not set the warnPartialMatchDollar option to TRUE.  In
>      >>> R-3.1.1, this was changed to not warn unless warnPartialMatchDollar was
>      >>> TRUE which was the PREVIOUS behaviour.  In other words, this change
>      >>> reverted the change made in R-3.1.0.  But instead of simply deleting
>      >>> the definition of $.data.frame, R-3.1.1 added extra code to it, the
>      >>> only effect of which is to slightly change the wording of the warning
>      >>> message from what is produced for any other list, while still retaining
>      >>> the massive slowdown.
>      >>> There is no need for you to write $.data.frame in C.  You just need
>      >>> to delete the version written in R.
>      >>
>      >> Sorry, I did misunderstand.  Thanks for the clarification.
>      >>
>      >> But if the "You" in your last sentence meant me, it needs to be "They": I am not a member of R Core and can't make any changes to the sources.
>      >>
>      >> Duncan Murdoch
>      >>
>      >> ______________________________________________
>      >> [hidden email] mailing list
>      >> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>      > --
>      > Peter Dalgaard, Professor,
>      > Center for Statistics, Copenhagen Business School
>      > Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>      > Phone: (+45)38153501
>      > Office: A 4.23
>      > Email: [hidden email]  Priv: [hidden email]
>
>      > ______________________________________________
>      > [hidden email] mailing list
>      > https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel