R Configuration Variable: Maximum Memory Allocation per R Instance

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

R Configuration Variable: Maximum Memory Allocation per R Instance

Juan Telleria
Dear R Developers,

In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the
innodb_buffer_pool_size or the key_buffer_size for setting the maximum
amount of RAM which can be used by a Server Instance:

¿Would it be possible to create an R Configuration Variable which fixes the
maximum amount of RAM memory to be used as Commit / Dynamic Memory
Allocation?

Thank you.
Juan

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R Configuration Variable: Maximum Memory Allocation per R Instance

Mark van der Loo
Dear Juan,

I'm not deeply familiar with the DB's you mention but it seems to me that
me that 'memory.limits' does what you want on one OS and you can use shell
commands to limit R's memory usage for *nix-alike systems (see
?memory.limits). Also, Jeroen Ooms wrote a nice article about this in the
JSS: https://www.jstatsoft.org/article/view/v055i07 . There's also a
package for it: RAppArmor.

-M




Op zo 17 sep. 2017 om 00:39 schreef Juan Telleria <[hidden email]>:

> Dear R Developers,
>
> In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the
> innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> amount of RAM which can be used by a Server Instance:
>
> ¿Would it be possible to create an R Configuration Variable which fixes the
> maximum amount of RAM memory to be used as Commit / Dynamic Memory
> Allocation?
>
> Thank you.
> Juan
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R Configuration Variable: Maximum Memory Allocation per R Instance

Juan Telleria
In reply to this post by Juan Telleria
This variables already exist as I have been said, and are:
* memory.size
* memory.limit

R Documentation:

https://stat.ethz.ch/R-manual/R-devel/library/utils/html/memory.size.html

Thank you,
Juan

El 17/9/2017 12:39 a. m., "Juan Telleria" <[hidden email]> escribió:

> Dear R Developers,
>
> In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the
> innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> amount of RAM which can be used by a Server Instance:
>
> ¿Would it be possible to create an R Configuration Variable which fixes
> the maximum amount of RAM memory to be used as Commit / Dynamic Memory
> Allocation?
>
> Thank you.
> Juan
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R Configuration Variable: Maximum Memory Allocation per R Instance

Jeroen Ooms
In reply to this post by Juan Telleria
On Sun, Sep 17, 2017 at 12:39 AM, Juan Telleria <[hidden email]> wrote:
> Dear R Developers,
>
> In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the
> innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> amount of RAM which can be used by a Server Instance.

Memory is not controlled by R itself because packages may malloc()
directly. However most operating systems have features to limit
resources of a given process. The CRAN package 'unix' has wrappers for
posix setrlimit [1] e.g. unix::rlimit_as() limits address space. This
works pretty well, however I found that the way memory is managed and
counted varies a lot per OS and malloc implementation.

You can also set rlimits on a single evaluation via the rlimit
parameter in sys::eval_safe().


[1] http://pubs.opengroup.org/onlinepubs/009695399/functions/getrlimit.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R Configuration Variable: Maximum Memory Allocation per R Instance

Juan Telleria
Very very interesting, if it is ok with it, I will post these observations
to Stack Overflow so that they are useful to other R Programmers, and make
more research on the topic, putting it all together. I could even do a
small article with my research.

I think this is a critical point if you want to have an R Instance and a
RDBMS in the same Server.

Thank you,
Juan

2017-09-17 14:14 GMT+02:00 Jeroen Ooms <[hidden email]>:

> On Sun, Sep 17, 2017 at 12:39 AM, Juan Telleria <[hidden email]>
> wrote:
> > Dear R Developers,
> >
> > In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have
> the
> > innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> > amount of RAM which can be used by a Server Instance.
>
> Memory is not controlled by R itself because packages may malloc()
> directly. However most operating systems have features to limit
> resources of a given process. The CRAN package 'unix' has wrappers for
> posix setrlimit [1] e.g. unix::rlimit_as() limits address space. This
> works pretty well, however I found that the way memory is managed and
> counted varies a lot per OS and malloc implementation.
>
> You can also set rlimits on a single evaluation via the rlimit
> parameter in sys::eval_safe().
>
>
> [1] http://pubs.opengroup.org/onlinepubs/009695399/
> functions/getrlimit.html
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel