R CMD check may not detect a code/documentation mismatch

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

R CMD check may not detect a code/documentation mismatch

Petr Savicky
For the package at
  http://www.cs.cas.cz/~savicky/R-devel/something_0.0.0.tar.gz
which is a minor part of some other package only to demonstrate the
problem, i get (under R version 2.11.0 Under development 2009-12-12 r50714
and also under R-2.9.2, openSUSE 11.1 (x86_64) and CentOS release 5.2)

  R CMD check something_0.0.0.tar.gz

  ...
  * checking Rd files ... OK
  * checking Rd metadata ... OK
  * checking Rd cross-references ... OK
  * checking for missing documentation entries ... OK
  * checking for code/documentation mismatches ... OK
  * checking Rd \usage sections ... OK
  * checking examples ... NONE
  * checking PDF version of manual ... OK

although the package code contains

  testCoreNA <- function()

and the documentation contains

  \usage{
  testCoreClass(verbose=0)
  testCoreAttrEval(verbose=0)
  testCoreReg(verbose=0)
  testCoreNA(verbose=0)
  }

There is a mismatch between code and documentation of testCoreNA(). Is the
problem caused by having four entries in \usage{} section?

Petr Savicky.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R CMD check may not detect a code/documentation mismatch

Peter Dalgaard
Petr Savicky wrote:

> For the package at
>   http://www.cs.cas.cz/~savicky/R-devel/something_0.0.0.tar.gz
> which is a minor part of some other package only to demonstrate the
> problem, i get (under R version 2.11.0 Under development 2009-12-12 r50714
> and also under R-2.9.2, openSUSE 11.1 (x86_64) and CentOS release 5.2)
>
>   R CMD check something_0.0.0.tar.gz
>
>   ...
>   * checking Rd files ... OK
>   * checking Rd metadata ... OK
>   * checking Rd cross-references ... OK
>   * checking for missing documentation entries ... OK
>   * checking for code/documentation mismatches ... OK
>   * checking Rd \usage sections ... OK
>   * checking examples ... NONE
>   * checking PDF version of manual ... OK
>
> although the package code contains
>
>   testCoreNA <- function()
>
> and the documentation contains
>
>   \usage{
>   testCoreClass(verbose=0)
>   testCoreAttrEval(verbose=0)
>   testCoreReg(verbose=0)
>   testCoreNA(verbose=0)
>   }
>
> There is a mismatch between code and documentation of testCoreNA(). Is the
> problem caused by having four entries in \usage{} section?

Hmm, looks more like a thinko in this code inside codoc():

     functions_in_code <- Filter(function(f) {
         f <- get(f, envir = code_env)
         is.function(f) && (length(formals(f)) > 0L)
     }, objects_in_code)

which, further down the line, causes functions with no formal arguments
to be skipped when compared to the usage section.

Browse[2]>
debug: ind <- (!functions %in% functions_to_be_ignored & functions %in%
     functions_in_code)
Browse[2]> functions
[1] "testCoreClass"    "testCoreAttrEval" "testCoreReg"
"testCoreNA"
Browse[2]>
debug: bad_functions <- mapply(functions[ind], exprs[ind], FUN =
function(x,
     y) check_codoc(x, as.pairlist(as.alist.call(y[-1L]))), SIMPLIFY =
FALSE)
Browse[2]> ind
[1]  TRUE  TRUE  TRUE FALSE

I.e. testCoreNA is never tested by check_codoc. There may of course be a
rationale for this, but it escapes me...


--
    O__  ---- Peter Dalgaard             Øster Farimagsgade 5, Entr.B
   c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
  (*) \(*) -- University of Copenhagen   Denmark      Ph:  (+45) 35327918
~~~~~~~~~~ - ([hidden email])              FAX: (+45) 35327907

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R CMD check may not detect a code/documentation mismatch

Kurt Hornik-5
>>>>> Peter Dalgaard writes:

> Petr Savicky wrote:
>> For the package at
>> http://www.cs.cas.cz/~savicky/R-devel/something_0.0.0.tar.gz
>> which is a minor part of some other package only to demonstrate the
>> problem, i get (under R version 2.11.0 Under development 2009-12-12 r50714
>> and also under R-2.9.2, openSUSE 11.1 (x86_64) and CentOS release 5.2)
>>
>> R CMD check something_0.0.0.tar.gz
>>
>> ...
>> * checking Rd files ... OK
>> * checking Rd metadata ... OK
>> * checking Rd cross-references ... OK
>> * checking for missing documentation entries ... OK
>> * checking for code/documentation mismatches ... OK
>> * checking Rd \usage sections ... OK
>> * checking examples ... NONE
>> * checking PDF version of manual ... OK
>>
>> although the package code contains
>>
>> testCoreNA <- function()
>>
>> and the documentation contains
>>
>> \usage{
>> testCoreClass(verbose=0)
>> testCoreAttrEval(verbose=0)
>> testCoreReg(verbose=0)
>> testCoreNA(verbose=0)
>> }
>>
>> There is a mismatch between code and documentation of testCoreNA(). Is the
>> problem caused by having four entries in \usage{} section?

> Hmm, looks more like a thinko in this code inside codoc():

>      functions_in_code <- Filter(function(f) {
>          f <- get(f, envir = code_env)
>          is.function(f) && (length(formals(f)) > 0L)
>      }, objects_in_code)

> which, further down the line, causes functions with no formal arguments
> to be skipped when compared to the usage section.

> Browse[2]>
> debug: ind <- (!functions %in% functions_to_be_ignored & functions %in%
>      functions_in_code)
> Browse[2]> functions
> [1] "testCoreClass"    "testCoreAttrEval" "testCoreReg"
> "testCoreNA"
> Browse[2]>
> debug: bad_functions <- mapply(functions[ind], exprs[ind], FUN =
> function(x,
>      y) check_codoc(x, as.pairlist(as.alist.call(y[-1L]))), SIMPLIFY =
> FALSE)
> Browse[2]> ind
> [1]  TRUE  TRUE  TRUE FALSE

> I.e. testCoreNA is never tested by check_codoc. There may of course be
> a rationale for this, but it escapes me...

Well, I am sure I had good reasons when I wrote the code many years ago,
but of course I no longer recall what they were.

Did you try the effect of removing the length(formals(f)) test?

-k


> --
>     O__  ---- Peter Dalgaard             Øster Farimagsgade 5, Entr.B
>    c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
>   (*) \(*) -- University of Copenhagen   Denmark      Ph:  (+45) 35327918
> ~~~~~~~~~~ - ([hidden email])              FAX: (+45) 35327907

> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R CMD check may not detect a code/documentation mismatch

Peter Dalgaard
Kurt Hornik wrote:

>>>>>> Peter Dalgaard writes:
>
>> Petr Savicky wrote:
>>> For the package at
>>> http://www.cs.cas.cz/~savicky/R-devel/something_0.0.0.tar.gz
>>> which is a minor part of some other package only to demonstrate the
>>> problem, i get (under R version 2.11.0 Under development 2009-12-12 r50714
>>> and also under R-2.9.2, openSUSE 11.1 (x86_64) and CentOS release 5.2)
>>>
>>> R CMD check something_0.0.0.tar.gz
>>>
>>> ...
>>> * checking Rd files ... OK
>>> * checking Rd metadata ... OK
>>> * checking Rd cross-references ... OK
>>> * checking for missing documentation entries ... OK
>>> * checking for code/documentation mismatches ... OK
>>> * checking Rd \usage sections ... OK
>>> * checking examples ... NONE
>>> * checking PDF version of manual ... OK
>>>
>>> although the package code contains
>>>
>>> testCoreNA <- function()
>>>
>>> and the documentation contains
>>>
>>> \usage{
>>> testCoreClass(verbose=0)
>>> testCoreAttrEval(verbose=0)
>>> testCoreReg(verbose=0)
>>> testCoreNA(verbose=0)
>>> }
>>>
>>> There is a mismatch between code and documentation of testCoreNA(). Is the
>>> problem caused by having four entries in \usage{} section?
>
>> Hmm, looks more like a thinko in this code inside codoc():
>
>>      functions_in_code <- Filter(function(f) {
>>          f <- get(f, envir = code_env)
>>          is.function(f) && (length(formals(f)) > 0L)
>>      }, objects_in_code)
>
>> which, further down the line, causes functions with no formal arguments
>> to be skipped when compared to the usage section.
>
>> Browse[2]>
>> debug: ind <- (!functions %in% functions_to_be_ignored & functions %in%
>>      functions_in_code)
>> Browse[2]> functions
>> [1] "testCoreClass"    "testCoreAttrEval" "testCoreReg"
>> "testCoreNA"
>> Browse[2]>
>> debug: bad_functions <- mapply(functions[ind], exprs[ind], FUN =
>> function(x,
>>      y) check_codoc(x, as.pairlist(as.alist.call(y[-1L]))), SIMPLIFY =
>> FALSE)
>> Browse[2]> ind
>> [1]  TRUE  TRUE  TRUE FALSE
>
>> I.e. testCoreNA is never tested by check_codoc. There may of course be
>> a rationale for this, but it escapes me...
>
> Well, I am sure I had good reasons when I wrote the code many years ago,
> but of course I no longer recall what they were.
>
> Did you try the effect of removing the length(formals(f)) test?

Not yet. Priorities....

-p

--
   O__  ---- Peter Dalgaard             Øster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark      Ph:  (+45) 35327918
~~~~~~~~~~ - ([hidden email])              FAX: (+45) 35327907

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R CMD check may not detect a code/documentation mismatch

Petr Savicky
In reply to this post by Kurt Hornik-5
On Mon, Dec 14, 2009 at 09:24:12AM +0100, Kurt Hornik wrote:
> >>>>> Peter Dalgaard writes:
[...]

> > Hmm, looks more like a thinko in this code inside codoc():
>
> >      functions_in_code <- Filter(function(f) {
> >          f <- get(f, envir = code_env)
> >          is.function(f) && (length(formals(f)) > 0L)
> >      }, objects_in_code)
>
> > which, further down the line, causes functions with no formal arguments
> > to be skipped when compared to the usage section.
>
> > Browse[2]>
> > debug: ind <- (!functions %in% functions_to_be_ignored & functions %in%
> >      functions_in_code)
> > Browse[2]> functions
> > [1] "testCoreClass"    "testCoreAttrEval" "testCoreReg"
> > "testCoreNA"
> > Browse[2]>
> > debug: bad_functions <- mapply(functions[ind], exprs[ind], FUN =
> > function(x,
> >      y) check_codoc(x, as.pairlist(as.alist.call(y[-1L]))), SIMPLIFY =
> > FALSE)
> > Browse[2]> ind
> > [1]  TRUE  TRUE  TRUE FALSE
>
> > I.e. testCoreNA is never tested by check_codoc. There may of course be
> > a rationale for this, but it escapes me...
>
> Well, I am sure I had good reasons when I wrote the code many years ago,
> but of course I no longer recall what they were.
>
> Did you try the effect of removing the length(formals(f)) test?

I understand this as the question, what is the influence of the following patch
(indented by two spaces).

  diff --minimal -U 3 -r R-devel_2009-12-13/src/library/tools/R/QC.R R-devel_length_formals/src/library/tools/R/QC.R
  --- R-devel_2009-12-13/src/library/tools/R/QC.R 2009-10-27 02:59:10.000000000 +0100
  +++ R-devel_length_formals/src/library/tools/R/QC.R 2009-12-14 18:45:55.000000000 +0100
  @@ -398,7 +398,7 @@
       functions_in_code <-
           Filter(function(f) {
                      f <- get(f, envir = code_env)
  -                   is.function(f) && (length(formals(f)) > 0L)
  +                   is.function(f)
                  },
                  objects_in_code)
       ## Sourcing all R code files in the package is a problem for base,

Since i do not know this part of R code, i simply applied the patch to the
current R-devel and run "make" and "make check" (both OK) and compared
the output of

  R CMD check XML_2.6-0.tar.gz
  R CMD check randomForest_4.5-33.tar.gz
  R CMD check tree_1.0-27.tar.gz
  R CMD check survival_2.35-7.tar.gz

with R being both original R-devel and the modified one. The results were identical
in all four cases (empty diff) on two Linux machines (CentOS and openSUSE).

On the other hand, for the package
  http://www.cs.cas.cz/~savicky/R-devel/something_0.0.0.tar.gz
which demonstrates the problem, the modified R detected the code/doc mismatch.
The diff of the two outputs of R CMD check was

  [savicky@uivtx test]$ diff original modified
  35c35,42
  < * checking for code/documentation mismatches ... OK
  ---
  > * checking for code/documentation mismatches ... WARNING
  > Codoc mismatches from documentation object 'testCore':
  > testCoreNA
  >   Code: function()
  >   Docs: function(verbose = 0)
  >   Argument names in docs not in code:
  >     verbose
  >
  39a47,50
  > WARNING: There was 1 warning, see
  >   /home/savicky/tmp/test/something.Rcheck/00check.log
  > for details
  >

Petr Savicky.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R CMD check may not detect a code/documentation mismatch

Peter Dalgaard
[orig. correspondence below]

Brian and Kurt have been sinking their teeth further into this. It
turned out that there was a gotcha with the .Primitive functions like
`:`, `.C`, etc., for which formals() always returns NULL. However,
replacement with a  check for typeof(f) == "closure" made things work,
and revealed a handful of undetected issues in CRAN packages.

So this has now been committed to r-devel. Thanks for chasing this up.

-pd


Petr Savicky wrote:

> On Mon, Dec 14, 2009 at 09:24:12AM +0100, Kurt Hornik wrote:
>>>>>>> Peter Dalgaard writes:
> [...]
>>> Hmm, looks more like a thinko in this code inside codoc():
>>>      functions_in_code <- Filter(function(f) {
>>>          f <- get(f, envir = code_env)
>>>          is.function(f) && (length(formals(f)) > 0L)
>>>      }, objects_in_code)
>>> which, further down the line, causes functions with no formal arguments
>>> to be skipped when compared to the usage section.
>>> Browse[2]>
>>> debug: ind <- (!functions %in% functions_to_be_ignored & functions %in%
>>>      functions_in_code)
>>> Browse[2]> functions
>>> [1] "testCoreClass"    "testCoreAttrEval" "testCoreReg"
>>> "testCoreNA"
>>> Browse[2]>
>>> debug: bad_functions <- mapply(functions[ind], exprs[ind], FUN =
>>> function(x,
>>>      y) check_codoc(x, as.pairlist(as.alist.call(y[-1L]))), SIMPLIFY =
>>> FALSE)
>>> Browse[2]> ind
>>> [1]  TRUE  TRUE  TRUE FALSE
>>> I.e. testCoreNA is never tested by check_codoc. There may of course be
>>> a rationale for this, but it escapes me...
>> Well, I am sure I had good reasons when I wrote the code many years ago,
>> but of course I no longer recall what they were.
>>
>> Did you try the effect of removing the length(formals(f)) test?
>
> I understand this as the question, what is the influence of the following patch
> (indented by two spaces).
>
>   diff --minimal -U 3 -r R-devel_2009-12-13/src/library/tools/R/QC.R R-devel_length_formals/src/library/tools/R/QC.R
>   --- R-devel_2009-12-13/src/library/tools/R/QC.R 2009-10-27 02:59:10.000000000 +0100
>   +++ R-devel_length_formals/src/library/tools/R/QC.R 2009-12-14 18:45:55.000000000 +0100
>   @@ -398,7 +398,7 @@
>        functions_in_code <-
>            Filter(function(f) {
>                       f <- get(f, envir = code_env)
>   -                   is.function(f) && (length(formals(f)) > 0L)
>   +                   is.function(f)
>                   },
>                   objects_in_code)
>        ## Sourcing all R code files in the package is a problem for base,
>
> Since i do not know this part of R code, i simply applied the patch to the
> current R-devel and run "make" and "make check" (both OK) and compared
> the output of
>
>   R CMD check XML_2.6-0.tar.gz
>   R CMD check randomForest_4.5-33.tar.gz
>   R CMD check tree_1.0-27.tar.gz
>   R CMD check survival_2.35-7.tar.gz
>
> with R being both original R-devel and the modified one. The results were identical
> in all four cases (empty diff) on two Linux machines (CentOS and openSUSE).
>
> On the other hand, for the package
>   http://www.cs.cas.cz/~savicky/R-devel/something_0.0.0.tar.gz
> which demonstrates the problem, the modified R detected the code/doc mismatch.
> The diff of the two outputs of R CMD check was
>
>   [savicky@uivtx test]$ diff original modified
>   35c35,42
>   < * checking for code/documentation mismatches ... OK
>   ---
>   > * checking for code/documentation mismatches ... WARNING
>   > Codoc mismatches from documentation object 'testCore':
>   > testCoreNA
>   >   Code: function()
>   >   Docs: function(verbose = 0)
>   >   Argument names in docs not in code:
>   >     verbose
>   >
>   39a47,50
>   > WARNING: There was 1 warning, see
>   >   /home/savicky/tmp/test/something.Rcheck/00check.log
>   > for details
>   >
>
> Petr Savicky.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel


--
   O__  ---- Peter Dalgaard             Øster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark      Ph:  (+45) 35327918
~~~~~~~~~~ - ([hidden email])              FAX: (+45) 35327907

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel