parallel computing in r....

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

parallel computing in r....

akshay kulkarni
dear members,
                            I am taking recourse to parallel computing to speed up my R code.

The mcparallel function(in parallel package),presumably, sends a single task (an R function, to be precise) to a single core. Is this right? Is that one and only one core? What if there are other cores lying idle?

As far as I am concerned, I am going to rent an AWS EC2 server with 48 cores. I have four functions to be parallelized. Can I send the thread of one function to 12 cores, so that all 48 cores are utilized? If so, how do I do that with mcparallel function in the parallel package?

Very many thanks for your time and effort...
yours sincerely....
AKSHAY M KULKARNI

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: parallel computing in r....

Jeff Newmiller
You cannot send one task to 12 processors... the price of parallelism is that you must break down your task into smaller tasks. Once you have number of tasks equal to our more than the number of available cores then the higher level functions such as parLapply or mclapply can shuffle tasks onto cores. If you have 48 cores and 4 tasks, only 4 of the cores will be used. You probably should stay away from directly calling mcparallel as it is a low level function. Read the vignette about the base "parallel" package [1]. You might also like to use the forEach contributed package to simplify your work, and should look at the CRAN High Performance Computing Task View [2].

[1] vignette("parallel") or http://stat.ethz.ch/R-manual/R-devel/library/parallel/doc/parallel.pdf
[2] https://cran.r-project.org/web/views/HighPerformanceComputing.html

On June 22, 2018 3:12:28 AM PDT, akshay kulkarni <[hidden email]> wrote:

>dear members,
>      I am taking recourse to parallel computing to speed up my R code.
>
>The mcparallel function(in parallel package),presumably, sends a single
>task (an R function, to be precise) to a single core. Is this right? Is
>that one and only one core? What if there are other cores lying idle?
>
>As far as I am concerned, I am going to rent an AWS EC2 server with 48
>cores. I have four functions to be parallelized. Can I send the thread
>of one function to 12 cores, so that all 48 cores are utilized? If so,
>how do I do that with mcparallel function in the parallel package?
>
>Very many thanks for your time and effort...
>yours sincerely....
>AKSHAY M KULKARNI
>
> [[alternative HTML version deleted]]
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: parallel computing in r....

Juan Telleria Ruiz de Aguirre
Maybe this R-bloggers post could also help you:

https://www.google.es/amp/s/www.r-bloggers.com/implementing-parallel-processing-in-r/amp/

https://www.google.es/amp/s/www.r-bloggers.com/a-guide-to-parallelism-in-r/amp/

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.