How to add make option to package compilation?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to add make option to package compilation?

Martin Møller Skarbiniks Pedersen
Hi,

  I am installing a lot of packages to a new R installation and it takes a
long time.
  However the machine got 4 cpus and most of the packages are written in
C/C++.

  So is it possible to add a -j4 flag to the make command when I use the
install.packages() function?
  That will probably speed up the package installation process 390%.

Regards
Martin M. S. Pedersen

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: How to add make option to package compilation?

Duncan Murdoch-2
On 15/09/2017 8:11 AM, Martin Møller Skarbiniks Pedersen wrote:

> Hi,
>
>    I am installing a lot of packages to a new R installation and it takes a
> long time.
>    However the machine got 4 cpus and most of the packages are written in
> C/C++.
>
>    So is it possible to add a -j4 flag to the make command when I use the
> install.packages() function?
>    That will probably speed up the package installation process 390%.

See the Ncpus argument in ?install.packages.

Duncan Murdoch

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: How to add make option to package compilation?

Martin Møller Skarbiniks Pedersen
On 15 September 2017 at 14:13, Duncan Murdoch <[hidden email]>
wrote:

> On 15/09/2017 8:11 AM, Martin Møller Skarbiniks Pedersen wrote:
>
>> Hi,
>>
>>    I am installing a lot of packages to a new R installation and it takes
>> a
>> long time.
>>    However the machine got 4 cpus and most of the packages are written in
>> C/C++.
>>
>>    So is it possible to add a -j4 flag to the make command when I use the
>> install.packages() function?
>>    That will probably speed up the package installation process 390%.
>>
>
> See the Ncpus argument in ?install.packages.


Thanks.

However it looks like Ncpus=4 tries to compile four R packages at the same
time using one cpu for each packages.

From the documentation:
"
   Ncpus: the number of parallel processes to use for a parallel
          install of more than one source package.  Values greater than
          one are supported if the ‘make’ command specified by
          ‘Sys.getenv("MAKE", "make")’ accepts argument ‘-k -j Ncpus’
"

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: How to add make option to package compilation?

Michael Dewey-3
In line

On 15/09/2017 13:30, Martin Møller Skarbiniks Pedersen wrote:

> On 15 September 2017 at 14:13, Duncan Murdoch <[hidden email]>
> wrote:
>
>> On 15/09/2017 8:11 AM, Martin Møller Skarbiniks Pedersen wrote:
>>
>>> Hi,
>>>
>>>     I am installing a lot of packages to a new R installation and it takes
>>> a
>>> long time.
>>>     However the machine got 4 cpus and most of the packages are written in
>>> C/C++.
>>>
>>>     So is it possible to add a -j4 flag to the make command when I use the
>>> install.packages() function?
>>>     That will probably speed up the package installation process 390%.
>>>
>>
>> See the Ncpus argument in ?install.packages.
>
>
> Thanks.
>
> However it looks like Ncpus=4 tries to compile four R packages at the same
> time using one cpu for each packages.
>

But you said you had lots to install so would that not speed things up too?

>  From the documentation:
> "
>     Ncpus: the number of parallel processes to use for a parallel
>            install of more than one source package.  Values greater than
>            one are supported if the ‘make’ command specified by
>            ‘Sys.getenv("MAKE", "make")’ accepts argument ‘-k -j Ncpus’
> "
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
>

--
Michael
http://www.dewey.myzen.co.uk/home.html

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: How to add make option to package compilation?

Martin Morgan-3
On 09/15/2017 08:57 AM, Michael Dewey wrote:

> In line
>
> On 15/09/2017 13:30, Martin Møller Skarbiniks Pedersen wrote:
>> On 15 September 2017 at 14:13, Duncan Murdoch <[hidden email]>
>> wrote:
>>
>>> On 15/09/2017 8:11 AM, Martin Møller Skarbiniks Pedersen wrote:
>>>
>>>> Hi,
>>>>
>>>>     I am installing a lot of packages to a new R installation and it
>>>> takes
>>>> a
>>>> long time.
>>>>     However the machine got 4 cpus and most of the packages are
>>>> written in
>>>> C/C++.
>>>>
>>>>     So is it possible to add a -j4 flag to the make command when I
>>>> use the
>>>> install.packages() function?
>>>>     That will probably speed up the package installation process 390%.
>>>>
>>>
>>> See the Ncpus argument in ?install.packages.
>>
>>
>> Thanks.
>>
>> However it looks like Ncpus=4 tries to compile four R packages at the
>> same
>> time using one cpu for each packages.

The variable MAKE is defined in ${R_HOME}/etc/Renviron, and can be
over-written with ~/.Renviron

     MAKE=make -j

There is further discussion in

 
https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Configuration-variables

and ?Renviron.

One could configure a source installation to always compile with make
-j, something like ./configure MAKE="make -j"

Martin

>>
>
> But you said you had lots to install so would that not speed things up too?
>
>>  From the documentation:
>> "
>>     Ncpus: the number of parallel processes to use for a parallel
>>            install of more than one source package.  Values greater than
>>            one are supported if the ‘make’ command specified by
>>            ‘Sys.getenv("MAKE", "make")’ accepts argument ‘-k -j Ncpus’
>> "
>>
>>     [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>> ---
>> This email has been checked for viruses by AVG.
>> http://www.avg.com
>>
>


This email message may contain legally privileged and/or...{{dropped:2}}

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.