Varying results of package checks due to random seed

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Varying results of package checks due to random seed

Radford Neal
> From: Philippe GROSJEAN <[hidden email]>
>
> ... for latest CRAN version, we have successfully installed 4999
> packages among the 5321 CRAN package on our platform. ... It is
> strange that a large portion of R CMD check errors on CRAN occur and
> disappear *without any version update* of a package or any of its
> direct or indirect dependencies! That is, a fraction of errors or
> warnings seem to appear and disappear without any code update.

Some of these are likely the result of packages running tests using
random number generation without setting the random numbers seed, in
which case the seed is set based on the current time and process id,
with an obvious possibility of results varying from run to run.

In the current development version of pqR (in branch 19-mods, found at
https://github.com/radfordneal/pqR/tree/19-mods), I have implemented a
change so that if the R_SEED environment variable is set, the random
seed is initialized to its value, rather than from the time and
process id.  This was motivated by exactly this problem - I can now
just set R_SEED to something before running all the package checks.

   Radford Neal

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Makevars clean target not in action?

randy3k
Hi, developers

When I run `R CMD INSTALL --clean my_package`, only the .o files and the .so file are removed, however, they were deleted not because of the clean target defined in Makevars.
The clean target was indeed not executed.

I have the following simple Marvars to show that clean target is not functioning.

Makevars
-----------------------------

all: foo $(SHLIB)

clean:
        rm -f *.o $(SHLIB)
        rm -f b

foo:
        touch b


After I run ``R CMD INSTALL --clean my_package``, the file `b` is still there.

Any thought?

Best,
Randy
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Varying results of package checks due to random seed

Paul Gilbert-2
In reply to this post by Radford Neal


On 03/22/2014 01:32 PM, Radford Neal wrote:

>> From: Philippe GROSJEAN <[hidden email]>
>>
>> ... for latest CRAN version, we have successfully installed 4999
>> packages among the 5321 CRAN package on our platform. ... It is
>> strange that a large portion of R CMD check errors on CRAN occur and
>> disappear *without any version update* of a package or any of its
>> direct or indirect dependencies! That is, a fraction of errors or
>> warnings seem to appear and disappear without any code update.
>
> Some of these are likely the result of packages running tests using
> random number generation without setting the random numbers seed, in
> which case the seed is set based on the current time and process id,
> with an obvious possibility of results varying from run to run.
>
> In the current development version of pqR (in branch 19-mods, found at
> https://github.com/radfordneal/pqR/tree/19-mods), I have implemented a
> change so that if the R_SEED environment variable is set, the random
> seed is initialized to its value, rather than from the time and
> process id.  This was motivated by exactly this problem - I can now
> just set R_SEED to something before running all the package checks.

Beware, if you are serious about reproducing things, that you really
need to save information about the uniform and other generators you use,
such as the normal generator. The defaults do not change often, but have
in the past, and could in the future if something better comes along.
There are some small utilities and examples in the package setRNG which
can help.

Also remember that you need to beware of a side effect of the
environment variable approach. It is great for reproducing things, as
you would want to do in package tests, but be careful how you use it in
functions as it may mess up the randomness if you always set the seed to
the same starting value.

Paul

>
>     Radford Neal
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel