PSOCK cluster and renice

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

PSOCK cluster and renice

Andreas Leha
Hi all,

Is it possible to use the 'renice' option together with parallel
clusters of type 'PSOCK'?  The help page for parallel::makeCluster is
not specific about which options are supported on which types and I am
getting the following message when passing renice = 19 :

> cl <- parallel::makeCluster(2, renice = 19)
nice: ‘+19’: No such file or directory

Kind regards,
Andreas

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: PSOCK cluster and renice

Henrik Bengtsson-5
Looks like a bug to me due to wrong assumptions about 'nice'
arguments, but could be because a "non-standard" 'nice' is used.  If
we do:

> trace(system, tracer = quote(print(command)))
Tracing function "system" in package "base"

we see that the system call used is:

> cl <- parallel::makePSOCKcluster(2L, renice = 19)
Tracing system(cmd, wait = FALSE) on entry
[1] "nice +19 '/usr/lib/R/bin/Rscript'
--default-packages=datasets,utils,grDevices,graphics,stats,methods -e
'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11146 OUT=/dev/null
TIMEOUT=2592000 XDR=TRUE"
nice: ‘+19’: No such file or directory
^C

The code that prepends that 'nice +19' is in parallel:::newPSOCKnode:

    if (!is.na(renice) && renice)
        cmd <- sprintf("nice +%d %s", as.integer(renice), cmd)

I don't know where that originates from and on what platform it was
tests/validated.  On Ubuntu 16.04, CentOS 6.6, and CentOS 7.4, I have
'nice' from "GNU coreutils" and they all complain about using '+',
e.g.

$ nice +19 date
nice: +19: No such file or directory

but '-' works:

$ nice -19 date
Sun Dec  3 20:01:31 PST 2017

Neither 'nice --help' nor 'man help' mention the use of a +n option.


WORKAROUND:  As a workaround, you can use:

cl <- future::makeClusterPSOCK(2L, rscript = c("nice",
"--adjustment=10", file.path(R.home("bin"), "Rscript")))

which is backward compatible with parallel::makePSOCKcluster() but
provides you with more detailed control.  Try adding verbose = TRUE to
see what the exact call looks like.

/Henrik


On Sun, Dec 3, 2017 at 7:35 PM, Andreas Leha
<[hidden email]> wrote:

> Hi all,
>
> Is it possible to use the 'renice' option together with parallel
> clusters of type 'PSOCK'?  The help page for parallel::makeCluster is
> not specific about which options are supported on which types and I am
> getting the following message when passing renice = 19 :
>
>> cl <- parallel::makeCluster(2, renice = 19)
> nice: ‘+19’: No such file or directory
>
> Kind regards,
> Andreas
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: PSOCK cluster and renice

Andreas Leha
Hi Henrik,

Thanks for the detailed in fast reply!

My guess would be that the confusion comes from the different use of nice and renice.

The workraund you provided work fine!  Thanks a lot.

Best,
Andreas



Henrik Bengtsson <[hidden email]> writes:

> Looks like a bug to me due to wrong assumptions about 'nice'
> arguments, but could be because a "non-standard" 'nice' is used.  If
> we do:
>
>> trace(system, tracer = quote(print(command)))
> Tracing function "system" in package "base"
>
> we see that the system call used is:
>
>> cl <- parallel::makePSOCKcluster(2L, renice = 19)
> Tracing system(cmd, wait = FALSE) on entry
> [1] "nice +19 '/usr/lib/R/bin/Rscript'
> --default-packages=datasets,utils,grDevices,graphics,stats,methods -e
> 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11146 OUT=/dev/null
> TIMEOUT=2592000 XDR=TRUE"
> nice: ‘+19’: No such file or directory
> ^C
>
> The code that prepends that 'nice +19' is in parallel:::newPSOCKnode:
>
>     if (!is.na(renice) && renice)
>         cmd <- sprintf("nice +%d %s", as.integer(renice), cmd)
>
> I don't know where that originates from and on what platform it was
> tests/validated.  On Ubuntu 16.04, CentOS 6.6, and CentOS 7.4, I have
> 'nice' from "GNU coreutils" and they all complain about using '+',
> e.g.
>
> $ nice +19 date
> nice: +19: No such file or directory
>
> but '-' works:
>
> $ nice -19 date
> Sun Dec  3 20:01:31 PST 2017
>
> Neither 'nice --help' nor 'man help' mention the use of a +n option.
>
>
> WORKAROUND:  As a workaround, you can use:
>
> cl <- future::makeClusterPSOCK(2L, rscript = c("nice",
> "--adjustment=10", file.path(R.home("bin"), "Rscript")))
>
> which is backward compatible with parallel::makePSOCKcluster() but
> provides you with more detailed control.  Try adding verbose = TRUE to
> see what the exact call looks like.
>
> /Henrik
>
>
> On Sun, Dec 3, 2017 at 7:35 PM, Andreas Leha
> <[hidden email]> wrote:
>> Hi all,
>>
>> Is it possible to use the 'renice' option together with parallel
>> clusters of type 'PSOCK'?  The help page for parallel::makeCluster is
>> not specific about which options are supported on which types and I am
>> getting the following message when passing renice = 19 :
>>
>>> cl <- parallel::makeCluster(2, renice = 19)
>> nice: ‘+19’: No such file or directory
>>
>> Kind regards,
>> Andreas
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.