makeCluster hangs

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

makeCluster hangs

T. Florian Jaeger-2
Hi all,

I can't get the functionality of the package parallel to work.
Specifically, makeCluster() hangs when I run it. I first noticed the
problem when trying to run Rstan with multiple cores and the traced it
back to the core package parallel. The following results in R hanging
after the call to makeCluster.

library(parallel)

# Calculate the number of cores
no_cores <- detectCores() - 1

# Initiate cluster
cl <- makeCluster(no_cores)

I'm running MacOS High Sierra 10.13.3 (17D47) on a MacbookPro 2017
laptop with 4 cores.

platform       x86_64-apple-darwin15.6.0
arch           x86_64
os             darwin15.6.0
system         x86_64, darwin15.6.0
status
major          3
minor          4.3
year           2017
month          11
day            30
svn rev        73796
language       R
version.string R version 3.4.3 (2017-11-30)
nickname       Kite-Eating Tree

The problem replicates in R --vanilla


I've spent hours googling for solutions but can't find any reports of
this problem. Any help would be appreciated.


Florian

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: makeCluster hangs

Henrik Bengtsson-5
A few quick comments:

* You mention R --vanilla, but make sure to try with
parallel::makeCluster(), so that you don't happen to pick up
snow::makeCluster() if 'snow' is attached and ahead of parallel on the
search() path.

* Try creating a single background worker, i.e. parallel::makeCluster(1L).

* Try with cl <- future::makeClusterPSOCK(1L, verbose = TRUE), which
gives the same thing, but it also show you some details on what it
does internally; that may give some clues where it stalls.

/Henrik

On Sat, Feb 10, 2018 at 12:11 PM, T. Florian Jaeger
<[hidden email]> wrote:

> Hi all,
>
> I can't get the functionality of the package parallel to work. Specifically,
> makeCluster() hangs when I run it. I first noticed the problem when trying
> to run Rstan with multiple cores and the traced it back to the core package
> parallel. The following results in R hanging after the call to makeCluster.
>
> library(parallel)
>
> # Calculate the number of cores
> no_cores <- detectCores() - 1
>
> # Initiate cluster
> cl <- makeCluster(no_cores)
>
> I'm running MacOS High Sierra 10.13.3 (17D47) on a MacbookPro 2017 laptop
> with 4 cores.
>
> platform       x86_64-apple-darwin15.6.0
> arch           x86_64
> os             darwin15.6.0
> system         x86_64, darwin15.6.0
> status
> major          3
> minor          4.3
> year           2017
> month          11
> day            30
> svn rev        73796
> language       R
> version.string R version 3.4.3 (2017-11-30)
> nickname       Kite-Eating Tree
>
> The problem replicates in R --vanilla
>
>
> I've spent hours googling for solutions but can't find any reports of this
> problem. Any help would be appreciated.
>
>
> Florian
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: makeCluster hangs

T. Florian Jaeger-2
Dear Henrik,

thank you, for the quick reply. Bizarrely enough, the problem vanished when
I woke the computer from sleep (I had previously replicated the problem
after several restarts of both R and the MacOS).

I will follow-up if I can again replicate the problem.

Florian


On 2/10/18 4:39 PM, Henrik Bengtsson wrote:

> A few quick comments:
>
> * You mention R --vanilla, but make sure to try with
> parallel::makeCluster(), so that you don't happen to pick up
> snow::makeCluster() if 'snow' is attached and ahead of parallel on the
> search() path.
>
> * Try creating a single background worker, i.e. parallel::makeCluster(1L).
>
> * Try with cl <- future::makeClusterPSOCK(1L, verbose = TRUE), which
> gives the same thing, but it also show you some details on what it
> does internally; that may give some clues where it stalls.
>
> /Henrik
>
> On Sat, Feb 10, 2018 at 12:11 PM, T. Florian Jaeger
> <[hidden email]> wrote:
>> Hi all,
>>
>> I can't get the functionality of the package parallel to work. Specifically,
>> makeCluster() hangs when I run it. I first noticed the problem when trying
>> to run Rstan with multiple cores and the traced it back to the core package
>> parallel. The following results in R hanging after the call to makeCluster.
>>
>> library(parallel)
>>
>> # Calculate the number of cores
>> no_cores <- detectCores() - 1
>>
>> # Initiate cluster
>> cl <- makeCluster(no_cores)
>>
>> I'm running MacOS High Sierra 10.13.3 (17D47) on a MacbookPro 2017 laptop
>> with 4 cores.
>>
>> platform       x86_64-apple-darwin15.6.0
>> arch           x86_64
>> os             darwin15.6.0
>> system         x86_64, darwin15.6.0
>> status
>> major          3
>> minor          4.3
>> year           2017
>> month          11
>> day            30
>> svn rev        73796
>> language       R
>> version.string R version 3.4.3 (2017-11-30)
>> nickname       Kite-Eating Tree
>>
>> The problem replicates in R --vanilla
>>
>>
>> I've spent hours googling for solutions but can't find any reports of this
>> problem. Any help would be appreciated.
>>
>>
>> Florian
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=O6dqVFPEDpdoXY3wkv8u6o0LHKx4WbQ_itn0O87jj5s&m=R7DIWWqYTP2xarhrvKcymtN3XlAQ9vHLFDhPL6FxQ60&s=7F8Lez-XE8iC2JBU4JYsEtF3U0HObhMnCASud5xTgNM&e=
> .
>

--
T. Florian Jaeger
Professor of the Brain and Cognitive Sciences,
Computer Science
Alfred P. Sloan Research Fellow
University of Rochester,
Meliora Hall, Box 270268,
Rochester, NY 14627-0268
USA
P: +1 (585) 276 3611
F: +1 (585) 442 9216
E: [hidden email]
B: http://www.hlplab.wordpress.com/
T: https://twitter.com/_hlplab_

To schedule an appointment, please visit
http://www.bcs.rochester.edu/people/fjaeger/appointments.html

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: makeCluster hangs

Tomas Kalibera
Also using R-devel might help - the forking support in parallel has been
made more robust against race conditions, but the changes are probably
too substantial to port to 3.4.x. If you find how to cause a race
condition using parallel/forking in R-devel, a report would be greatly
appreciated.

Tomas

On 02/11/2018 09:51 PM, T. Florian Jaeger wrote:

> Dear Henrik,
>
> thank you, for the quick reply. Bizarrely enough, the problem vanished when
> I woke the computer from sleep (I had previously replicated the problem
> after several restarts of both R and the MacOS).
>
> I will follow-up if I can again replicate the problem.
>
> Florian
>
>
> On 2/10/18 4:39 PM, Henrik Bengtsson wrote:
>> A few quick comments:
>>
>> * You mention R --vanilla, but make sure to try with
>> parallel::makeCluster(), so that you don't happen to pick up
>> snow::makeCluster() if 'snow' is attached and ahead of parallel on the
>> search() path.
>>
>> * Try creating a single background worker, i.e. parallel::makeCluster(1L).
>>
>> * Try with cl <- future::makeClusterPSOCK(1L, verbose = TRUE), which
>> gives the same thing, but it also show you some details on what it
>> does internally; that may give some clues where it stalls.
>>
>> /Henrik
>>
>> On Sat, Feb 10, 2018 at 12:11 PM, T. Florian Jaeger
>> <[hidden email]> wrote:
>>> Hi all,
>>>
>>> I can't get the functionality of the package parallel to work. Specifically,
>>> makeCluster() hangs when I run it. I first noticed the problem when trying
>>> to run Rstan with multiple cores and the traced it back to the core package
>>> parallel. The following results in R hanging after the call to makeCluster.
>>>
>>> library(parallel)
>>>
>>> # Calculate the number of cores
>>> no_cores <- detectCores() - 1
>>>
>>> # Initiate cluster
>>> cl <- makeCluster(no_cores)
>>>
>>> I'm running MacOS High Sierra 10.13.3 (17D47) on a MacbookPro 2017 laptop
>>> with 4 cores.
>>>
>>> platform       x86_64-apple-darwin15.6.0
>>> arch           x86_64
>>> os             darwin15.6.0
>>> system         x86_64, darwin15.6.0
>>> status
>>> major          3
>>> minor          4.3
>>> year           2017
>>> month          11
>>> day            30
>>> svn rev        73796
>>> language       R
>>> version.string R version 3.4.3 (2017-11-30)
>>> nickname       Kite-Eating Tree
>>>
>>> The problem replicates in R --vanilla
>>>
>>>
>>> I've spent hours googling for solutions but can't find any reports of this
>>> problem. Any help would be appreciated.
>>>
>>>
>>> Florian
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Ddevel&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=O6dqVFPEDpdoXY3wkv8u6o0LHKx4WbQ_itn0O87jj5s&m=R7DIWWqYTP2xarhrvKcymtN3XlAQ9vHLFDhPL6FxQ60&s=7F8Lez-XE8iC2JBU4JYsEtF3U0HObhMnCASud5xTgNM&e=
>> .
>>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel