Crash when installing heavy packages in remote server with R and Rstudio server

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Crash when installing heavy packages in remote server with R and Rstudio server

Jorge Cimentada
Hi,

I've been trying to install some very heavy packages from Github (~ 100MB -
300 MB) on my remote server using Rstudio server and I keep getting some
crashes. These packages are pretty much datasets.

I try doing it with this code and get the error below.

> install.packages("devtools")

> devtools::install_github("pbiecek/PISA2000lite")
Downloading GitHub repo pbiecek/PISA2000lite@master
from URL https://api.github.com/repos/pbiecek/PISA2000lite/zipball/master
Installing PISA2000lite
'/usr/lib/R/bin/R' --no-site-file --no-environ --no-save --no-restore
--quiet CMD INSTALL  \
  '/tmp/Rtmp1GoUVG/devtools76a72d179d7/pbiecek-PISA2000lite-54f4765'  \
  --library='/home/cimentadaj/R/x86_64-pc-linux-gnu-library/3.4'
--install-tests

* installing *source* package ‘PISA2000lite’ ...
** data
*** moving datasets to lazyload DB
Killed
Installation failed: Command failed (137)



The package actually gets installed
any(grepl("PISA2012lite", installed.packages()[, 1]))

[1] TRUE


But I once I load the library it's as if none of the datasets are
available. I've used these packages for a long time on my local computer so
I'm sure it's not the packages. The server has enough space to handle the
packages, so it's not memory either. Below you can find the specs of the R
session and the server


R version 3.4.1 (2017-06-30)

Platform: x86_64-pc-linux-gnu (64-bit)

Running under: Ubuntu 16.04.3 LTS

Matrix products: default

BLAS: /usr/lib/libblas/libblas.so.3.6.0

LAPACK: /usr/lib/lapack/liblapack.so.3.6.0


locale:

 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
LC_TIME=en_US.UTF-8

 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8
 LC_MESSAGES=en_US.UTF-8

 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C


[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=C


attached base packages:

[1] stats     graphics  grDevices utils     datasets  methods   base


loaded via a namespace (and not attached):

 [1] httr_1.2.1      compiler_3.4.1  R6_2.2.2        tools_3.4.1
withr_2.0.0     curl_2.8.1

 [7] memoise_1.1.0   git2r_0.19.0    digest_0.6.12   devtools_1.13.3


The disk info:


Filesystem      Size  Used Avail Use% Mounted on

udev            981M     0  981M   0% /dev

tmpfs           201M   21M  180M  11% /run

/dev/vda1        39G  9.1G   30G  24% /

tmpfs          1001M  840K 1000M   1% /dev/shm

tmpfs           5.0M     0  5.0M   0% /run/lock

tmpfs          1001M     0 1001M   0% /sys/fs/cgroup

/dev/vda15      105M  3.4M  102M   4% /boot/efi

tmpfs           201M     0  201M   0% /run/user/1000

And server specs from Digital Ocean:
2 GB Memory / 40 GB Disk / LON1 - Ubuntu 16.04.3 x64 from

I posted the question in this
<https://stackoverflow.com/questions/45522145/crash-when-installing-heavy-packages-in-remote-server-with-rstudio-server?noredirect=1#comment78011638_45522145>
stackoverflow thread and actually found out that the problem is not from
Rstudio server in itself because I get the exact same error with:
R CMD INSTALL PISA2000lite_1.0.tar.gz

Any idea why this might be occurring?

-----------------------------------

Jorge Cimentada

*https://cimentadaj.github.io/ <https://cimentadaj.github.io/>*

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Crash when installing heavy packages in remote server with R and Rstudio server

Duncan Murdoch-2
On 06/08/2017 9:17 AM, Jorge Cimentada wrote:
> Hi,
>
> I've been trying to install some very heavy packages from Github (~ 100MB -
> 300 MB) on my remote server using Rstudio server and I keep getting some
> crashes. These packages are pretty much datasets.

Based on your description, this is RStudio Server specific.  You'll need
to write to their tech support for help.  This list is for R issues.

Duncan Murdoch

>
> I try doing it with this code and get the error below.
>
>> install.packages("devtools")
>
>> devtools::install_github("pbiecek/PISA2000lite")
> Downloading GitHub repo pbiecek/PISA2000lite@master
> from URL https://api.github.com/repos/pbiecek/PISA2000lite/zipball/master
> Installing PISA2000lite
> '/usr/lib/R/bin/R' --no-site-file --no-environ --no-save --no-restore
> --quiet CMD INSTALL  \
>   '/tmp/Rtmp1GoUVG/devtools76a72d179d7/pbiecek-PISA2000lite-54f4765'  \
>   --library='/home/cimentadaj/R/x86_64-pc-linux-gnu-library/3.4'
> --install-tests
>
> * installing *source* package ‘PISA2000lite’ ...
> ** data
> *** moving datasets to lazyload DB
> Killed
> Installation failed: Command failed (137)
>
>
>
> The package actually gets installed
> any(grepl("PISA2012lite", installed.packages()[, 1]))
>
> [1] TRUE
>
>
> But I once I load the library it's as if none of the datasets are
> available. I've used these packages for a long time on my local computer so
> I'm sure it's not the packages. The server has enough space to handle the
> packages, so it's not memory either. Below you can find the specs of the R
> session and the server
>
>
> R version 3.4.1 (2017-06-30)
>
> Platform: x86_64-pc-linux-gnu (64-bit)
>
> Running under: Ubuntu 16.04.3 LTS
>
> Matrix products: default
>
> BLAS: /usr/lib/libblas/libblas.so.3.6.0
>
> LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
>
>
> locale:
>
>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
> LC_TIME=en_US.UTF-8
>
>  [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8
>  LC_MESSAGES=en_US.UTF-8
>
>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C
>
>
> [10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8
> LC_IDENTIFICATION=C
>
>
> attached base packages:
>
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
>
> loaded via a namespace (and not attached):
>
>  [1] httr_1.2.1      compiler_3.4.1  R6_2.2.2        tools_3.4.1
> withr_2.0.0     curl_2.8.1
>
>  [7] memoise_1.1.0   git2r_0.19.0    digest_0.6.12   devtools_1.13.3
>
>
> The disk info:
>
>
> Filesystem      Size  Used Avail Use% Mounted on
>
> udev            981M     0  981M   0% /dev
>
> tmpfs           201M   21M  180M  11% /run
>
> /dev/vda1        39G  9.1G   30G  24% /
>
> tmpfs          1001M  840K 1000M   1% /dev/shm
>
> tmpfs           5.0M     0  5.0M   0% /run/lock
>
> tmpfs          1001M     0 1001M   0% /sys/fs/cgroup
>
> /dev/vda15      105M  3.4M  102M   4% /boot/efi
>
> tmpfs           201M     0  201M   0% /run/user/1000
>
> And server specs from Digital Ocean:
> 2 GB Memory / 40 GB Disk / LON1 - Ubuntu 16.04.3 x64 from
>
> I posted the question in this
> <https://stackoverflow.com/questions/45522145/crash-when-installing-heavy-packages-in-remote-server-with-rstudio-server?noredirect=1#comment78011638_45522145>
> stackoverflow thread and actually found out that the problem is not from
> Rstudio server in itself because I get the exact same error with:
> R CMD INSTALL PISA2000lite_1.0.tar.gz
>
> Any idea why this might be occurring?
>
> -----------------------------------
>
> Jorge Cimentada
>
> *https://cimentadaj.github.io/ <https://cimentadaj.github.io/>*
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Crash when installing heavy packages in remote server with R and Rstudio server

Duncan Murdoch-2
On 06/08/2017 2:57 PM, Duncan Murdoch wrote:
> On 06/08/2017 9:17 AM, Jorge Cimentada wrote:
>> Hi,
>>
>> I've been trying to install some very heavy packages from Github (~ 100MB -
>> 300 MB) on my remote server using Rstudio server and I keep getting some
>> crashes. These packages are pretty much datasets.
>
> Based on your description, this is RStudio Server specific.  You'll need
> to write to their tech support for help.  This list is for R issues.

Oops, I see below that you found it happens with plain R as well.  Was
that tested on your server machine as well?   It looks as though it has
some execution time limit (or disk or memory limit) and you're exceeding
it.  So you should talk to your sysadmin.

Duncan Murdoch

>
> Duncan Murdoch
>
>>
>> I try doing it with this code and get the error below.
>>
>>> install.packages("devtools")
>>
>>> devtools::install_github("pbiecek/PISA2000lite")
>> Downloading GitHub repo pbiecek/PISA2000lite@master
>> from URL https://api.github.com/repos/pbiecek/PISA2000lite/zipball/master
>> Installing PISA2000lite
>> '/usr/lib/R/bin/R' --no-site-file --no-environ --no-save --no-restore
>> --quiet CMD INSTALL  \
>>   '/tmp/Rtmp1GoUVG/devtools76a72d179d7/pbiecek-PISA2000lite-54f4765'  \
>>   --library='/home/cimentadaj/R/x86_64-pc-linux-gnu-library/3.4'
>> --install-tests
>>
>> * installing *source* package ‘PISA2000lite’ ...
>> ** data
>> *** moving datasets to lazyload DB
>> Killed
>> Installation failed: Command failed (137)
>>
>>
>>
>> The package actually gets installed
>> any(grepl("PISA2012lite", installed.packages()[, 1]))
>>
>> [1] TRUE
>>
>>
>> But I once I load the library it's as if none of the datasets are
>> available. I've used these packages for a long time on my local computer so
>> I'm sure it's not the packages. The server has enough space to handle the
>> packages, so it's not memory either. Below you can find the specs of the R
>> session and the server
>>
>>
>> R version 3.4.1 (2017-06-30)
>>
>> Platform: x86_64-pc-linux-gnu (64-bit)
>>
>> Running under: Ubuntu 16.04.3 LTS
>>
>> Matrix products: default
>>
>> BLAS: /usr/lib/libblas/libblas.so.3.6.0
>>
>> LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
>>
>>
>> locale:
>>
>>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>> LC_TIME=en_US.UTF-8
>>
>>  [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8
>>  LC_MESSAGES=en_US.UTF-8
>>
>>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C
>>
>>
>> [10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8
>> LC_IDENTIFICATION=C
>>
>>
>> attached base packages:
>>
>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>
>>
>> loaded via a namespace (and not attached):
>>
>>  [1] httr_1.2.1      compiler_3.4.1  R6_2.2.2        tools_3.4.1
>> withr_2.0.0     curl_2.8.1
>>
>>  [7] memoise_1.1.0   git2r_0.19.0    digest_0.6.12   devtools_1.13.3
>>
>>
>> The disk info:
>>
>>
>> Filesystem      Size  Used Avail Use% Mounted on
>>
>> udev            981M     0  981M   0% /dev
>>
>> tmpfs           201M   21M  180M  11% /run
>>
>> /dev/vda1        39G  9.1G   30G  24% /
>>
>> tmpfs          1001M  840K 1000M   1% /dev/shm
>>
>> tmpfs           5.0M     0  5.0M   0% /run/lock
>>
>> tmpfs          1001M     0 1001M   0% /sys/fs/cgroup
>>
>> /dev/vda15      105M  3.4M  102M   4% /boot/efi
>>
>> tmpfs           201M     0  201M   0% /run/user/1000
>>
>> And server specs from Digital Ocean:
>> 2 GB Memory / 40 GB Disk / LON1 - Ubuntu 16.04.3 x64 from
>>
>> I posted the question in this
>> <https://stackoverflow.com/questions/45522145/crash-when-installing-heavy-packages-in-remote-server-with-rstudio-server?noredirect=1#comment78011638_45522145>
>> stackoverflow thread and actually found out that the problem is not from
>> Rstudio server in itself because I get the exact same error with:
>> R CMD INSTALL PISA2000lite_1.0.tar.gz
>>
>> Any idea why this might be occurring?
>>
>> -----------------------------------
>>
>> Jorge Cimentada
>>
>> *https://cimentadaj.github.io/ <https://cimentadaj.github.io/>*
>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.