running R with users home dirs on a shared filesystems

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

running R with users home dirs on a shared filesystems

R devel mailing list
Hi guys,

I want to ask devel for who knows better - having multiple
nodes serving users home dirs off the same shared network
filesystem : are there any precautions or must-dos &
must-donts in order to assure healthy and efficient parallel
Rs running simultaneously - and I don't mean obvious stuff,
I'm rather asking about R's internals & environment.

simple example: three nodes mount a NFS share and users on
all three nodes run R simultaneously.

many thanks, L.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: running R with users home dirs on a shared filesystems

Simon Urbanek
User home is not used by R directly, so it is really up to whatever package/code may be using user home. In our setup we have all machines using NFS mounted homes for years. From experience the only thing to watch for are packages that use their own cache directories in $HOME instead of tempdir() - it is technically against CRAN policies but we have seen it in the wild.

Cheers,
Simon



> On Dec 13, 2019, at 1:36 PM, lejeczek via R-devel <[hidden email]> wrote:
>
> Hi guys,
>
> I want to ask devel for who knows better - having multiple
> nodes serving users home dirs off the same shared network
> filesystem : are there any precautions or must-dos &
> must-donts in order to assure healthy and efficient parallel
> Rs running simultaneously - and I don't mean obvious stuff,
> I'm rather asking about R's internals & environment.
>
> simple example: three nodes mount a NFS share and users on
> all three nodes run R simultaneously.
>
> many thanks, L.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: running R with users home dirs on a shared filesystems

Cook, Malcolm
Another thing to avoid are having multiple processes simultaneously access single sqlite3 database stored on NFS mount.

From sqlite manual: “Your best defense is to not use SQLite for files on a network filesystem<https://www.sqlite.org/lockingv3.html>”

So, if you configuring RStudio Server, make sure to follow advice about RStudio Package Manager<https://docs.rstudio.com/rspm/admin/database.html>: “This location must exist on local storage”

And any package that uses sqlite “under the hood” will similarly want the db on local storage to avoid such issues stemming from multi-process access.

Cheers,
Malcolm

From: R-devel <[hidden email]> On Behalf Of Simon Urbanek
Sent: Friday, December 13, 2019 12:52 PM
To: lejeczek <[hidden email]>
Cc: r-devel <[hidden email]>
Subject: Re: [Rd] running R with users home dirs on a shared filesystems

CAUTION: This email was received from an External Source


User home is not used by R directly, so it is really up to whatever package/code may be using user home. In our setup we have all machines using NFS mounted homes for years. From experience the only thing to watch for are packages that use their own cache directories in $HOME instead of tempdir() - it is technically against CRAN policies but we have seen it in the wild.

Cheers,
Simon



> On Dec 13, 2019, at 1:36 PM, lejeczek via R-devel <[hidden email]<mailto:[hidden email]>> wrote:
>
> Hi guys,
>
> I want to ask devel for who knows better - having multiple
> nodes serving users home dirs off the same shared network
> filesystem : are there any precautions or must-dos &
> must-donts in order to assure healthy and efficient parallel
> Rs running simultaneously - and I don't mean obvious stuff,
> I'm rather asking about R's internals & environment.
>
> simple example: three nodes mount a NFS share and users on
> all three nodes run R simultaneously.
>
> many thanks, L.
>
> ______________________________________________
> [hidden email]<mailto:[hidden email]> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email]<mailto:[hidden email]> mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel