Error: memory exhausted (limit reached?)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Error: memory exhausted (limit reached?)

Guillaume MULLER
Hello everyone,

I was working on a DeepNet project at home, with an old PC with 4Gb RAM, running Ubuntu 16.04.

For efficiency reason, I stored my dataset as a csv file with write.csv and reloaded it at will with read.csv. I did it several time, everything was working fine.

This morning, at work, I wanted to use a more powerful machine with 8Gb of RAM running under Ubuntu 16.10, but I ran into a strange error:

$ R
16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
Error: memory exhausted (limit reached?)
Error: C stack usage  7970548 is too close to the limit


I read a few fora on the Internet and found a potential workaround, consisting in increasing the stack size using ulimit. Unfortunately, it doesn't work for me:
$ ulimit -s
8192
$ ulimit -s $((100*$(ulimit -s)))
$ R --vanilla
16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
Error: memory exhausted (limit reached?)

How is it possible that a machine with twice the RAM as the previous one cannot load my file? Also how is it possible that a 513MB file cannot be read on a machine with 8GB RAM?

Since the only other difference is the Ubuntu version (thus R version), I assume there's a bug, but I don't know where to look for...

-----------------
FYI, my R version is:


16:05:12 R > version
               _
platform       x86_64-pc-linux-gnu
arch           x86_64
os             linux-gnu
system         x86_64, linux-gnu
status
major          3
minor          3.1
year           2016
month          06
day            21
svn rev        70800
language       R
version.string R version 3.3.1 (2016-06-21)
nickname       Bug in Your Hair

-----------------
For the sake of completeness, my Trainset is available at the following link, looks perfectly correct and loads perfectly @home but not @work...

https://mega.nz/#!1E9VHYKZ!isyoJEe6zoLdtC3efoZQreyxKP8sVJyvaiiLSEg4sBY

$ ls trainSetWhole.csv
-rw------- 1 guize guize 513M Mar  1 18:14 trainSetWhole.csv
$ head -n2 trainSetWhole.csv
"","i1","i2","i3","i4","i5","i6","i7","i8","i9","i10","i11","i12","i13","i14","i15","i16","i17","i18","i19","i20","i21","i22","i23","i24","i25","i26","i27","i28","i29","i30","i31","i32","i33","i34","i35","i36","i37","i38","i39","i40","i41","i42","i43","i44","i45","i46","i47","i48","i49","i50","i51","i52","i53","i54","i55","i56","i57","i58","i59","i60","i61","i62","i63","i64","i65","i66","i67","i68","i69","i70","i71","i72","i73","i74","i75","i76","i77","i78","i79","i80","i81","i82","i83","i84","i85","i86","i87","i88","i89","i90","i91","i92","i93","i94","i95","i96","i97","i98","i99","i100","i101","i102","i103","i104","i105","i106","i107","i108","i109","i110","i111","i112","i113","i114","i115","i116","i117","i118","i119","i120","i121","i122","i123","i124","i125","i126","i127","i128","i129","i130","i131","i132","i133","i134","i135","i136","i137","i138","i139","i140","c0","c1","c2","c3","c4","c5","c6","c7","c8","c9","cE","cH"
"1",-1.37446682201354e-17,2.26278846107692e-17,-1.27974510570192e-17,0.350805746453904,0.0689877720064749,3.96782166297516e-17,8.3747934622645e-17,-3.10693828550825e-17,-6.20727662108592e-17,3.50781139850846e-17,-7.43520083579608e-18,-2.01937105782818e-17,7.90527666844222e-17,0.00144287421147209,-1.88026404914112e-17,7.50553543898089e-17,-3.1156305544842e-17,3.33522787760855e-17,-8.2365725541805e-17,5.58356055978426e-17,-1.0212113811016e-17,2.03979863594455e-17,4.09717807934463e-17,0.0216467781249343,-3.44153670483002e-17,1.63751844321319e-17,0.0370002406507207,6.07678690807364e-18,-8.21680252405594e-17,4.44832722506604e-17,6.4069604073711e-17,-2.25481427040111e-17,4.03090616886684e-17,0.0326641123208996,-7.66887661628525e-17,2.74073501177539e-19,2.16820561358683e-17,4.48746042635995e-17,-1.09133528709867e-16,7.27192988615857e-17,3.12504939940947e-17,1.00863400203073e-17,-1.16417253776312e-17,0.000827131700928926,2.15591878205102e-17,-3.55400897372337e-17,-5.31549197577629e-!
 17,6.36151166067713e-17,-3.98986076789542e-17,8.923132020282e-17,2.25307349483604e-17,1.12105833438192e-17,6.11627660475528e-17,0.00457304914240373,-4.40071366400508e-17,-2.44832622161246e-17,0.0421621636081912,2.46626049639688e-17,-9.62286119146992e-18,-1.88510143728912e-17,-1.04308238059585e-17,-2.93643045718137e-17,2.20818728169428e-17,-4.44567220452202e-17,-8.94603852191165e-17,-9.94874448913085e-18,0.0114804462249642,-9.35428456683928e-18,-1.80861616310379e-17,3.92469708878433e-17,-4.74071809520974e-17,-9.12321682896516e-18,-3.69120306784775e-17,0.13411638484663,-1.15868506960443e-16,-7.46389966104001e-17,0.015082205973855,6.53743919930211e-18,2.67372672961429e-17,-1.91214022579557e-17,-2.63284305395001e-18,4.36662713526659e-17,-1.32415731622945e-17,0.0917760638890958,-5.1463921187737e-17,-3.06212083550592e-17,0.020354034217623,1.42110160178925e-17,8.78127934370195e-18,1.10296552176504e-17,6.04875461249026e-18,-1.12907950679241e-16,6.41934790659652e-17,0.06359065323963!
 94,-5.80721386923163e-17,2.60888831084575e-17,0.318234894813079,-2.644
05232759225e-17,-3.83483045548365e-17,6.03901669028453e-17,-8.12491526717919e-17,-2.31961766473706e-17,2.49851938854807e-17,0.0666134385177111,0.277994494373684,0.757276941042661,-6.17404037289403e-18,7.89863674358793e-17,-7.67835382820615e-17,1.02140113616298e-17,1.52661784797379e-17,-9.56651701308766e-17,-3.02020715735595e-17,0.388119618133219,0.407682070148667,2.2374985383075e-18,1.11172424441548e-16,5.834694549909e-18,-6.98992457434969e-17,2.58373931894799e-17,1.77047896799026e-17,-1.32411436947046e-17,2.48508545648632e-17,0.00261698558797486,-4.74012929746027e-18,-6.88567774177344e-18,-1.42368729782461e-17,3.86907870454172e-18,-1.18115468072844e-16,5.8232650855203e-17,-6.19221175303288e-18,1.66357706932964e-17,6.39953697342706e-17,2.06819618922065e-17,1.12542559622211e-17,1.61613122517958e-17,-8.54903096963431e-18,-2.91678323773648e-17,-1.24859483206721e-16,7.55251863839618e-18,1,0,0,0,0,0,0,0,0,0,0,0


If anyone has an idea...
______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Fwd: Error: memory exhausted (limit reached?)

Guillaume MULLER
Hi again,

Sorry, I messed up, the link to the actual file is:
https://mega.nz/#!ZMs0TSRJ!47DCZCnE6_FnICUp8MVS2R9eY_GdVIyGZ5O9TiejHfc

FYI, it loads perfectly on 2 machines with Ubuntu 16.04 and R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree"

But exceededs stack mem under Ubuntu 16.10's R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"


-------- Forwarded Message --------
Subject: Error: memory exhausted (limit reached?)
Date: Mon, 13 Mar 2017 16:51:15 +0100
From: Guillaume MULLER <[hidden email]>
To: [hidden email]

Hello everyone,

I was working on a DeepNet project at home, with an old PC with 4Gb RAM, running Ubuntu 16.04.

For efficiency reason, I stored my dataset as a csv file with write.csv and reloaded it at will with read.csv. I did it several time, everything was working fine.

This morning, at work, I wanted to use a more powerful machine with 8Gb of RAM running under Ubuntu 16.10, but I ran into a strange error:

$ R
16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
Error: memory exhausted (limit reached?)
Error: C stack usage  7970548 is too close to the limit


I read a few fora on the Internet and found a potential workaround, consisting in increasing the stack size using ulimit. Unfortunately, it doesn't work for me:
$ ulimit -s
8192
$ ulimit -s $((100*$(ulimit -s)))
$ R --vanilla
16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
Error: memory exhausted (limit reached?)

How is it possible that a machine with twice the RAM as the previous one cannot load my file? Also how is it possible that a 513MB file cannot be read on a machine with 8GB RAM?

Since the only other difference is the Ubuntu version (thus R version), I assume there's a bug, but I don't know where to look for...

-----------------
FYI, my R version is:


16:05:12 R > version
               _
platform       x86_64-pc-linux-gnu
arch           x86_64
os             linux-gnu
system         x86_64, linux-gnu
status
major          3
minor          3.1
year           2016
month          06
day            21
svn rev        70800
language       R
version.string R version 3.3.1 (2016-06-21)
nickname       Bug in Your Hair

-----------------
For the sake of completeness, my Trainset is available at the following link, looks perfectly correct and loads perfectly @home but not @work...

https://mega.nz/#!1E9VHYKZ!isyoJEe6zoLdtC3efoZQreyxKP8sVJyvaiiLSEg4sBY

$ ls trainSetWhole.csv
-rw------- 1 guize guize 513M Mar  1 18:14 trainSetWhole.csv
$ head -n2 trainSetWhole.csv
"","i1","i2","i3","i4","i5","i6","i7","i8","i9","i10","i11","i12","i13","i14","i15","i16","i17","i18","i19","i20","i21","i22","i23","i24","i25","i26","i27","i28","i29","i30","i31","i32","i33","i34","i35","i36","i37","i38","i39","i40","i41","i42","i43","i44","i45","i46","i47","i48","i49","i50","i51","i52","i53","i54","i55","i56","i57","i58","i59","i60","i61","i62","i63","i64","i65","i66","i67","i68","i69","i70","i71","i72","i73","i74","i75","i76","i77","i78","i79","i80","i81","i82","i83","i84","i85","i86","i87","i88","i89","i90","i91","i92","i93","i94","i95","i96","i97","i98","i99","i100","i101","i102","i103","i104","i105","i106","i107","i108","i109","i110","i111","i112","i113","i114","i115","i116","i117","i118","i119","i120","i121","i122","i123","i124","i125","i126","i127","i128","i129","i130","i131","i132","i133","i134","i135","i136","i137","i138","i139","i140","c0","c1","c2","c3","c4","c5","c6","c7","c8","c9","cE","cH"
"1",-1.37446682201354e-17,2.26278846107692e-17,-1.27974510570192e-17,0.350805746453904,0.0689877720064749,3.96782166297516e-17,8.3747934622645e-17,-3.10693828550825e-17,-6.20727662108592e-17,3.50781139850846e-17,-7.43520083579608e-18,-2.01937105782818e-17,7.90527666844222e-17,0.00144287421147209,-1.88026404914112e-17,7.50553543898089e-17,-3.1156305544842e-17,3.33522787760855e-17,-8.2365725541805e-17,5.58356055978426e-17,-1.0212113811016e-17,2.03979863594455e-17,4.09717807934463e-17,0.0216467781249343,-3.44153670483002e-17,1.63751844321319e-17,0.0370002406507207,6.07678690807364e-18,-8.21680252405594e-17,4.44832722506604e-17,6.4069604073711e-17,-2.25481427040111e-17,4.03090616886684e-17,0.0326641123208996,-7.66887661628525e-17,2.74073501177539e-19,2.16820561358683e-17,4.48746042635995e-17,-1.09133528709867e-16,7.27192988615857e-17,3.12504939940947e-17,1.00863400203073e-17,-1.16417253776312e-17,0.000827131700928926,2.15591878205102e-17,-3.55400897372337e-17,-5.31549197577629e-!
 17,6.36151166067713e-17,-3.98986076789542e-17,8.923132020282e-17,2.25307349483604e-17,1.12105833438192e-17,6.11627660475528e-17,0.00457304914240373,-4.40071366400508e-17,-2.44832622161246e-17,0.0421621636081912,2.46626049639688e-17,-9.62286119146992e-18,-1.88510143728912e-17,-1.04308238059585e-17,-2.93643045718137e-17,2.20818728169428e-17,-4.44567220452202e-17,-8.94603852191165e-17,-9.94874448913085e-18,0.0114804462249642,-9.35428456683928e-18,-1.80861616310379e-17,3.92469708878433e-17,-4.74071809520974e-17,-9.12321682896516e-18,-3.69120306784775e-17,0.13411638484663,-1.15868506960443e-16,-7.46389966104001e-17,0.015082205973855,6.53743919930211e-18,2.67372672961429e-17,-1.91214022579557e-17,-2.63284305395001e-18,4.36662713526659e-17,-1.32415731622945e-17,0.0917760638890958,-5.1463921187737e-17,-3.06212083550592e-17,0.020354034217623,1.42110160178925e-17,8.78127934370195e-18,1.10296552176504e-17,6.04875461249026e-18,-1.12907950679241e-16,6.41934790659652e-17,0.06359065323963!
 94,-5.80721386923163e-17,2.60888831084575e-17,0.318234894813079,-2.644
05232759225e-17,-3.83483045548365e-17,6.03901669028453e-17,-8.12491526717919e-17,-2.31961766473706e-17,2.49851938854807e-17,0.0666134385177111,0.277994494373684,0.757276941042661,-6.17404037289403e-18,7.89863674358793e-17,-7.67835382820615e-17,1.02140113616298e-17,1.52661784797379e-17,-9.56651701308766e-17,-3.02020715735595e-17,0.388119618133219,0.407682070148667,2.2374985383075e-18,1.11172424441548e-16,5.834694549909e-18,-6.98992457434969e-17,2.58373931894799e-17,1.77047896799026e-17,-1.32411436947046e-17,2.48508545648632e-17,0.00261698558797486,-4.74012929746027e-18,-6.88567774177344e-18,-1.42368729782461e-17,3.86907870454172e-18,-1.18115468072844e-16,5.8232650855203e-17,-6.19221175303288e-18,1.66357706932964e-17,6.39953697342706e-17,2.06819618922065e-17,1.12542559622211e-17,1.61613122517958e-17,-8.54903096963431e-18,-2.91678323773648e-17,-1.24859483206721e-16,7.55251863839618e-18,1,0,0,0,0,0,0,0,0,0,0,0


If anyone has an idea...
______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fwd: Error: memory exhausted (limit reached?)

Bert Gunter-2
I suspect this is beyond the expertise of most participants of this
list. You might wish to post this on r-devel instead,where such
expertise is more likely to be found. However, that also may not be
the right place either; it's just the best I could think of.

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Mar 14, 2017 at 2:42 AM, Guillaume MULLER <[hidden email]> wrote:

> Hi again,
>
> Sorry, I messed up, the link to the actual file is:
> https://mega.nz/#!ZMs0TSRJ!47DCZCnE6_FnICUp8MVS2R9eY_GdVIyGZ5O9TiejHfc
>
> FYI, it loads perfectly on 2 machines with Ubuntu 16.04 and R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree"
>
> But exceededs stack mem under Ubuntu 16.10's R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
>
>
> -------- Forwarded Message --------
> Subject: Error: memory exhausted (limit reached?)
> Date: Mon, 13 Mar 2017 16:51:15 +0100
> From: Guillaume MULLER <[hidden email]>
> To: [hidden email]
>
> Hello everyone,
>
> I was working on a DeepNet project at home, with an old PC with 4Gb RAM, running Ubuntu 16.04.
>
> For efficiency reason, I stored my dataset as a csv file with write.csv and reloaded it at will with read.csv. I did it several time, everything was working fine.
>
> This morning, at work, I wanted to use a more powerful machine with 8Gb of RAM running under Ubuntu 16.10, but I ran into a strange error:
>
> $ R
> 16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
> Error: memory exhausted (limit reached?)
> Error: C stack usage  7970548 is too close to the limit
>
>
> I read a few fora on the Internet and found a potential workaround, consisting in increasing the stack size using ulimit. Unfortunately, it doesn't work for me:
> $ ulimit -s
> 8192
> $ ulimit -s $((100*$(ulimit -s)))
> $ R --vanilla
> 16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
> Error: memory exhausted (limit reached?)
>
> How is it possible that a machine with twice the RAM as the previous one cannot load my file? Also how is it possible that a 513MB file cannot be read on a machine with 8GB RAM?
>
> Since the only other difference is the Ubuntu version (thus R version), I assume there's a bug, but I don't know where to look for...
>
> -----------------
> FYI, my R version is:
>
>
> 16:05:12 R > version
>                _
> platform       x86_64-pc-linux-gnu
> arch           x86_64
> os             linux-gnu
> system         x86_64, linux-gnu
> status
> major          3
> minor          3.1
> year           2016
> month          06
> day            21
> svn rev        70800
> language       R
> version.string R version 3.3.1 (2016-06-21)
> nickname       Bug in Your Hair
>
> -----------------
> For the sake of completeness, my Trainset is available at the following link, looks perfectly correct and loads perfectly @home but not @work...
>
> https://mega.nz/#!1E9VHYKZ!isyoJEe6zoLdtC3efoZQreyxKP8sVJyvaiiLSEg4sBY
>
> $ ls trainSetWhole.csv
> -rw------- 1 guize guize 513M Mar  1 18:14 trainSetWhole.csv
> $ head -n2 trainSetWhole.csv
> "","i1","i2","i3","i4","i5","i6","i7","i8","i9","i10","i11","i12","i13","i14","i15","i16","i17","i18","i19","i20","i21","i22","i23","i24","i25","i26","i27","i28","i29","i30","i31","i32","i33","i34","i35","i36","i37","i38","i39","i40","i41","i42","i43","i44","i45","i46","i47","i48","i49","i50","i51","i52","i53","i54","i55","i56","i57","i58","i59","i60","i61","i62","i63","i64","i65","i66","i67","i68","i69","i70","i71","i72","i73","i74","i75","i76","i77","i78","i79","i80","i81","i82","i83","i84","i85","i86","i87","i88","i89","i90","i91","i92","i93","i94","i95","i96","i97","i98","i99","i100","i101","i102","i103","i104","i105","i106","i107","i108","i109","i110","i111","i112","i113","i114","i115","i116","i117","i118","i119","i120","i121","i122","i123","i124","i125","i126","i127","i128","i129","i130","i131","i132","i133","i134","i135","i136","i137","i138","i139","i140","c0","c1","c2","c3","c4","c5","c6","c7","c8","c9","cE","cH"
> "1",-1.37446682201354e-17,2.26278846107692e-17,-1.27974510570192e-17,0.350805746453904,0.0689877720064749,3.96782166297516e-17,8.3747934622645e-17,-3.10693828550825e-17,-6.20727662108592e-17,3.50781139850846e-17,-7.43520083579608e-18,-2.01937105782818e-17,7.90527666844222e-17,0.00144287421147209,-1.88026404914112e-17,7.50553543898089e-17,-3.1156305544842e-17,3.33522787760855e-17,-8.2365725541805e-17,5.58356055978426e-17,-1.0212113811016e-17,2.03979863594455e-17,4.09717807934463e-17,0.0216467781249343,-3.44153670483002e-17,1.63751844321319e-17,0.0370002406507207,6.07678690807364e-18,-8.21680252405594e-17,4.44832722506604e-17,6.4069604073711e-17,-2.25481427040111e-17,4.03090616886684e-17,0.0326641123208996,-7.66887661628525e-17,2.74073501177539e-19,2.16820561358683e-17,4.48746042635995e-17,-1.09133528709867e-16,7.27192988615857e-17,3.12504939940947e-17,1.00863400203073e-17,-1.16417253776312e-17,0.000827131700928926,2.15591878205102e-17,-3.55400897372337e-17,-5.31549197577629!
 e-!
>  17,6.36151166067713e-17,-3.98986076789542e-17,8.923132020282e-17,2.25307349483604e-17,1.12105833438192e-17,6.11627660475528e-17,0.00457304914240373,-4.40071366400508e-17,-2.44832622161246e-17,0.0421621636081912,2.46626049639688e-17,-9.62286119146992e-18,-1.88510143728912e-17,-1.04308238059585e-17,-2.93643045718137e-17,2.20818728169428e-17,-4.44567220452202e-17,-8.94603852191165e-17,-9.94874448913085e-18,0.0114804462249642,-9.35428456683928e-18,-1.80861616310379e-17,3.92469708878433e-17,-4.74071809520974e-17,-9.12321682896516e-18,-3.69120306784775e-17,0.13411638484663,-1.15868506960443e-16,-7.46389966104001e-17,0.015082205973855,6.53743919930211e-18,2.67372672961429e-17,-1.91214022579557e-17,-2.63284305395001e-18,4.36662713526659e-17,-1.32415731622945e-17,0.0917760638890958,-5.1463921187737e-17,-3.06212083550592e-17,0.020354034217623,1.42110160178925e-17,8.78127934370195e-18,1.10296552176504e-17,6.04875461249026e-18,-1.12907950679241e-16,6.41934790659652e-17,0.063590653239!
 63!

>  94,-5.80721386923163e-17,2.60888831084575e-17,0.318234894813079,-2.644
> 05232759225e-17,-3.83483045548365e-17,6.03901669028453e-17,-8.12491526717919e-17,-2.31961766473706e-17,2.49851938854807e-17,0.0666134385177111,0.277994494373684,0.757276941042661,-6.17404037289403e-18,7.89863674358793e-17,-7.67835382820615e-17,1.02140113616298e-17,1.52661784797379e-17,-9.56651701308766e-17,-3.02020715735595e-17,0.388119618133219,0.407682070148667,2.2374985383075e-18,1.11172424441548e-16,5.834694549909e-18,-6.98992457434969e-17,2.58373931894799e-17,1.77047896799026e-17,-1.32411436947046e-17,2.48508545648632e-17,0.00261698558797486,-4.74012929746027e-18,-6.88567774177344e-18,-1.42368729782461e-17,3.86907870454172e-18,-1.18115468072844e-16,5.8232650855203e-17,-6.19221175303288e-18,1.66357706932964e-17,6.39953697342706e-17,2.06819618922065e-17,1.12542559622211e-17,1.61613122517958e-17,-8.54903096963431e-18,-2.91678323773648e-17,-1.24859483206721e-16,7.55251863839618e-18,1,0,0,0,0,0,0,0,0,0,0,0
>
>
> If anyone has an idea...
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fwd: Error: memory exhausted (limit reached?)

Spencer Graves-4


On 2017-03-14 12:01 PM, Bert Gunter wrote:

> I suspect this is beyond the expertise of most participants of this
> list. You might wish to post this on r-devel instead,where such
> expertise is more likely to be found. However, that also may not be
> the right place either; it's just the best I could think of.
>
> Cheers,
> Bert
>
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along
> and sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Tue, Mar 14, 2017 at 2:42 AM, Guillaume MULLER <[hidden email]> wrote:
>> Hi again,
>>
>> Sorry, I messed up, the link to the actual file is:
>> https://mega.nz/#!ZMs0TSRJ!47DCZCnE6_FnICUp8MVS2R9eY_GdVIyGZ5O9TiejHfc
>>
>> FYI, it loads perfectly on 2 machines with Ubuntu 16.04 and R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree"
>>
>> But exceededs stack mem under Ubuntu 16.10's R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"


       Did you try R 3.3.3?  Spencer Graves

>>
>> -------- Forwarded Message --------
>> Subject: Error: memory exhausted (limit reached?)
>> Date: Mon, 13 Mar 2017 16:51:15 +0100
>> From: Guillaume MULLER <[hidden email]>
>> To: [hidden email]
>>
>> Hello everyone,
>>
>> I was working on a DeepNet project at home, with an old PC with 4Gb RAM, running Ubuntu 16.04.
>>
>> For efficiency reason, I stored my dataset as a csv file with write.csv and reloaded it at will with read.csv. I did it several time, everything was working fine.
>>
>> This morning, at work, I wanted to use a more powerful machine with 8Gb of RAM running under Ubuntu 16.10, but I ran into a strange error:
>>
>> $ R
>> 16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
>> Error: memory exhausted (limit reached?)
>> Error: C stack usage  7970548 is too close to the limit
>>
>>
>> I read a few fora on the Internet and found a potential workaround, consisting in increasing the stack size using ulimit. Unfortunately, it doesn't work for me:
>> $ ulimit -s
>> 8192
>> $ ulimit -s $((100*$(ulimit -s)))
>> $ R --vanilla
>> 16:05:12 R > trainSet <- read.csv("trainSetWhole.csv")
>> Error: memory exhausted (limit reached?)
>>
>> How is it possible that a machine with twice the RAM as the previous one cannot load my file? Also how is it possible that a 513MB file cannot be read on a machine with 8GB RAM?
>>
>> Since the only other difference is the Ubuntu version (thus R version), I assume there's a bug, but I don't know where to look for...
>>
>> -----------------
>> FYI, my R version is:
>>
>>
>> 16:05:12 R > version
>>                 _
>> platform       x86_64-pc-linux-gnu
>> arch           x86_64
>> os             linux-gnu
>> system         x86_64, linux-gnu
>> status
>> major          3
>> minor          3.1
>> year           2016
>> month          06
>> day            21
>> svn rev        70800
>> language       R
>> version.string R version 3.3.1 (2016-06-21)
>> nickname       Bug in Your Hair
>>
>> -----------------
>> For the sake of completeness, my Trainset is available at the following link, looks perfectly correct and loads perfectly @home but not @work...
>>
>> https://mega.nz/#!1E9VHYKZ!isyoJEe6zoLdtC3efoZQreyxKP8sVJyvaiiLSEg4sBY
>>
>> $ ls trainSetWhole.csv
>> -rw------- 1 guize guize 513M Mar  1 18:14 trainSetWhole.csv
>> $ head -n2 trainSetWhole.csv
>> "","i1","i2","i3","i4","i5","i6","i7","i8","i9","i10","i11","i12","i13","i14","i15","i16","i17","i18","i19","i20","i21","i22","i23","i24","i25","i26","i27","i28","i29","i30","i31","i32","i33","i34","i35","i36","i37","i38","i39","i40","i41","i42","i43","i44","i45","i46","i47","i48","i49","i50","i51","i52","i53","i54","i55","i56","i57","i58","i59","i60","i61","i62","i63","i64","i65","i66","i67","i68","i69","i70","i71","i72","i73","i74","i75","i76","i77","i78","i79","i80","i81","i82","i83","i84","i85","i86","i87","i88","i89","i90","i91","i92","i93","i94","i95","i96","i97","i98","i99","i100","i101","i102","i103","i104","i105","i106","i107","i108","i109","i110","i111","i112","i113","i114","i115","i116","i117","i118","i119","i120","i121","i122","i123","i124","i125","i126","i127","i128","i129","i130","i131","i132","i133","i134","i135","i136","i137","i138","i139","i140","c0","c1","c2","c3","c4","c5","c6","c7","c8","c9","cE","cH"
>> "1",-1.37446682201354e-17,2.26278846107692e-17,-1.27974510570192e-17,0.350805746453904,0.0689877720064749,3.96782166297516e-17,8.3747934622645e-17,-3.10693828550825e-17,-6.20727662108592e-17,3.50781139850846e-17,-7.43520083579608e-18,-2.01937105782818e-17,7.90527666844222e-17,0.00144287421147209,-1.88026404914112e-17,7.50553543898089e-17,-3.1156305544842e-17,3.33522787760855e-17,-8.2365725541805e-17,5.58356055978426e-17,-1.0212113811016e-17,2.03979863594455e-17,4.09717807934463e-17,0.0216467781249343,-3.44153670483002e-17,1.63751844321319e-17,0.0370002406507207,6.07678690807364e-18,-8.21680252405594e-17,4.44832722506604e-17,6.4069604073711e-17,-2.25481427040111e-17,4.03090616886684e-17,0.0326641123208996,-7.66887661628525e-17,2.74073501177539e-19,2.16820561358683e-17,4.48746042635995e-17,-1.09133528709867e-16,7.27192988615857e-17,3.12504939940947e-17,1.00863400203073e-17,-1.16417253776312e-17,0.000827131700928926,2.15591878205102e-17,-3.55400897372337e-17,-5.3154919757762!
 9!
>   e-!
>>   17,6.36151166067713e-17,-3.98986076789542e-17,8.923132020282e-17,2.25307349483604e-17,1.12105833438192e-17,6.11627660475528e-17,0.00457304914240373,-4.40071366400508e-17,-2.44832622161246e-17,0.0421621636081912,2.46626049639688e-17,-9.62286119146992e-18,-1.88510143728912e-17,-1.04308238059585e-17,-2.93643045718137e-17,2.20818728169428e-17,-4.44567220452202e-17,-8.94603852191165e-17,-9.94874448913085e-18,0.0114804462249642,-9.35428456683928e-18,-1.80861616310379e-17,3.92469708878433e-17,-4.74071809520974e-17,-9.12321682896516e-18,-3.69120306784775e-17,0.13411638484663,-1.15868506960443e-16,-7.46389966104001e-17,0.015082205973855,6.53743919930211e-18,2.67372672961429e-17,-1.91214022579557e-17,-2.63284305395001e-18,4.36662713526659e-17,-1.32415731622945e-17,0.0917760638890958,-5.1463921187737e-17,-3.06212083550592e-17,0.020354034217623,1.42110160178925e-17,8.78127934370195e-18,1.10296552176504e-17,6.04875461249026e-18,-1.12907950679241e-16,6.41934790659652e-17,0.0635906532!
 39!

>   63!
>>   94,-5.80721386923163e-17,2.60888831084575e-17,0.318234894813079,-2.644
>> 05232759225e-17,-3.83483045548365e-17,6.03901669028453e-17,-8.12491526717919e-17,-2.31961766473706e-17,2.49851938854807e-17,0.0666134385177111,0.277994494373684,0.757276941042661,-6.17404037289403e-18,7.89863674358793e-17,-7.67835382820615e-17,1.02140113616298e-17,1.52661784797379e-17,-9.56651701308766e-17,-3.02020715735595e-17,0.388119618133219,0.407682070148667,2.2374985383075e-18,1.11172424441548e-16,5.834694549909e-18,-6.98992457434969e-17,2.58373931894799e-17,1.77047896799026e-17,-1.32411436947046e-17,2.48508545648632e-17,0.00261698558797486,-4.74012929746027e-18,-6.88567774177344e-18,-1.42368729782461e-17,3.86907870454172e-18,-1.18115468072844e-16,5.8232650855203e-17,-6.19221175303288e-18,1.66357706932964e-17,6.39953697342706e-17,2.06819618922065e-17,1.12542559622211e-17,1.61613122517958e-17,-8.54903096963431e-18,-2.91678323773648e-17,-1.24859483206721e-16,7.55251863839618e-18,1,0,0,0,0,0,0,0,0,0,0,0
>>
>>
>> If anyone has an idea...
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Loading...