mcmapply Core Usage

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

mcmapply Core Usage

Dario Strbenac-2
Hello,

I have a minimal example that shows a problem I'm having with parallel processing.

library(parallel)
mcmapply(function(x, y)
{
  print("Running")
  Sys.sleep(10)
}, as.list(1:10), as.list(10:1), mc.cores = 16, SIMPLIFY = FALSE)

I see "Running" printed once every ten seconds. I read the documentation for mcmapply, but I don't understand why it wouldn't print all ten output statements almost at once, then wait for about ten seconds, then return to the prompt. Is it documented in another location ?

My problem is that I have a section of code that takes a long time, which I have replaced with Sys.sleep in the above example, but when I look at the output of top, I only see 1 CPU being used. The computer has 16 processors.

> sessionInfo()
R version 3.1.0 (2014-04-10)
Platform: x86_64-pc-linux-gnu (64-bit)

locale:
 [1] LC_CTYPE=en_AU.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_AU.UTF-8        LC_COLLATE=en_AU.UTF-8    
 [5] LC_MONETARY=en_AU.UTF-8    LC_MESSAGES=en_AU.UTF-8  
 [7] LC_PAPER=en_AU.UTF-8       LC_NAME=C                
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C      

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base    

loaded via a namespace (and not attached):
[1] tools_3.1.0

--------------------------------------
Dario Strbenac
PhD Student
University of Sydney
Camperdown NSW 2050
Australia

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: mcmapply Core Usage

Simon Urbanek
Dario,

yes, that is a bug, now fixed.

Cheers,
Simon


On May 29, 2014, at 2:00 AM, Dario Strbenac <[hidden email]> wrote:

> Hello,
>
> I have a minimal example that shows a problem I'm having with parallel processing.
>
> library(parallel)
> mcmapply(function(x, y)
> {
>  print("Running")
>  Sys.sleep(10)
> }, as.list(1:10), as.list(10:1), mc.cores = 16, SIMPLIFY = FALSE)
>
> I see "Running" printed once every ten seconds. I read the documentation for mcmapply, but I don't understand why it wouldn't print all ten output statements almost at once, then wait for about ten seconds, then return to the prompt. Is it documented in another location ?
>
> My problem is that I have a section of code that takes a long time, which I have replaced with Sys.sleep in the above example, but when I look at the output of top, I only see 1 CPU being used. The computer has 16 processors.
>
>> sessionInfo()
> R version 3.1.0 (2014-04-10)
> Platform: x86_64-pc-linux-gnu (64-bit)
>
> locale:
> [1] LC_CTYPE=en_AU.UTF-8       LC_NUMERIC=C              
> [3] LC_TIME=en_AU.UTF-8        LC_COLLATE=en_AU.UTF-8    
> [5] LC_MONETARY=en_AU.UTF-8    LC_MESSAGES=en_AU.UTF-8  
> [7] LC_PAPER=en_AU.UTF-8       LC_NAME=C                
> [9] LC_ADDRESS=C               LC_TELEPHONE=C            
> [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C      
>
> attached base packages:
> [1] parallel  stats     graphics  grDevices utils     datasets  methods  
> [8] base    
>
> loaded via a namespace (and not attached):
> [1] tools_3.1.0
>
> --------------------------------------
> Dario Strbenac
> PhD Student
> University of Sydney
> Camperdown NSW 2050
> Australia
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: mcmapply Core Usage

Henric Winell
Simon Urbanek skrev 2014-05-29 15:06:
> Dario,
>
> yes, that is a bug, now fixed.

Thanks for fixing it.

This problem was independently reported some time ago
(https://bugs.r-project.org/bugzilla/show_bug.cgi?id=15016) and I
believe that it can now be closed.

Henric


>
> Cheers,
> Simon
>
>
> On May 29, 2014, at 2:00 AM, Dario Strbenac <[hidden email]> wrote:
>
>> Hello,
>>
>> I have a minimal example that shows a problem I'm having with parallel processing.
>>
>> library(parallel)
>> mcmapply(function(x, y)
>> {
>>   print("Running")
>>   Sys.sleep(10)
>> }, as.list(1:10), as.list(10:1), mc.cores = 16, SIMPLIFY = FALSE)
>>
>> I see "Running" printed once every ten seconds. I read the documentation for mcmapply, but I don't understand why it wouldn't print all ten output statements almost at once, then wait for about ten seconds, then return to the prompt. Is it documented in another location ?
>>
>> My problem is that I have a section of code that takes a long time, which I have replaced with Sys.sleep in the above example, but when I look at the output of top, I only see 1 CPU being used. The computer has 16 processors.
>>
>>> sessionInfo()
>> R version 3.1.0 (2014-04-10)
>> Platform: x86_64-pc-linux-gnu (64-bit)
>>
>> locale:
>> [1] LC_CTYPE=en_AU.UTF-8       LC_NUMERIC=C
>> [3] LC_TIME=en_AU.UTF-8        LC_COLLATE=en_AU.UTF-8
>> [5] LC_MONETARY=en_AU.UTF-8    LC_MESSAGES=en_AU.UTF-8
>> [7] LC_PAPER=en_AU.UTF-8       LC_NAME=C
>> [9] LC_ADDRESS=C               LC_TELEPHONE=C
>> [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C
>>
>> attached base packages:
>> [1] parallel  stats     graphics  grDevices utils     datasets  methods
>> [8] base
>>
>> loaded via a namespace (and not attached):
>> [1] tools_3.1.0
>>
>> --------------------------------------
>> Dario Strbenac
>> PhD Student
>> University of Sydney
>> Camperdown NSW 2050
>> Australia
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: mcmapply Core Usage

Simon Urbanek
On May 29, 2014, at 8:04 PM, Henric Winell <[hidden email]> wrote:

> Simon Urbanek skrev 2014-05-29 15:06:
>> Dario,
>>
>> yes, that is a bug, now fixed.
>
> Thanks for fixing it.
>
> This problem was independently reported some time ago (https://bugs.r-project.org/bugzilla/show_bug.cgi?id=15016) and I believe that it can now be closed.
>

Ah, thanks, done,
Simon


>
>>
>> Cheers,
>> Simon
>>
>>
>> On May 29, 2014, at 2:00 AM, Dario Strbenac <[hidden email]> wrote:
>>
>>> Hello,
>>>
>>> I have a minimal example that shows a problem I'm having with parallel processing.
>>>
>>> library(parallel)
>>> mcmapply(function(x, y)
>>> {
>>>  print("Running")
>>>  Sys.sleep(10)
>>> }, as.list(1:10), as.list(10:1), mc.cores = 16, SIMPLIFY = FALSE)
>>>
>>> I see "Running" printed once every ten seconds. I read the documentation for mcmapply, but I don't understand why it wouldn't print all ten output statements almost at once, then wait for about ten seconds, then return to the prompt. Is it documented in another location ?
>>>
>>> My problem is that I have a section of code that takes a long time, which I have replaced with Sys.sleep in the above example, but when I look at the output of top, I only see 1 CPU being used. The computer has 16 processors.
>>>
>>>> sessionInfo()
>>> R version 3.1.0 (2014-04-10)
>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>
>>> locale:
>>> [1] LC_CTYPE=en_AU.UTF-8       LC_NUMERIC=C
>>> [3] LC_TIME=en_AU.UTF-8        LC_COLLATE=en_AU.UTF-8
>>> [5] LC_MONETARY=en_AU.UTF-8    LC_MESSAGES=en_AU.UTF-8
>>> [7] LC_PAPER=en_AU.UTF-8       LC_NAME=C
>>> [9] LC_ADDRESS=C               LC_TELEPHONE=C
>>> [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C
>>>
>>> attached base packages:
>>> [1] parallel  stats     graphics  grDevices utils     datasets  methods
>>> [8] base
>>>
>>> loaded via a namespace (and not attached):
>>> [1] tools_3.1.0
>>>
>>> --------------------------------------
>>> Dario Strbenac
>>> PhD Student
>>> University of Sydney
>>> Camperdown NSW 2050
>>> Australia
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel