Rprof causing R to crash

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Rprof causing R to crash

Marian Talbert
I'm trying to use Rprof() to identify bottlenecks and speed up a particullary slow section of code which reads in a portion of a tif file and compares each of the values to values of predictors used for model fitting.  I've written up an example that anyone can run.  Generally temp would be a section of a tif read into a data.frame and used later for other processing.  The first portion which just records the time works in about 6 seconds the second part causes RGui to immediately close with no error or warning.  Any advice on how to get Rprof to work or how to speed up this code would be greatly appreciated.  I'm using Windows 7 (which might be my problem) and R version 2.15.0.  

CalcMESS<-function(tiff.entry,pred.vect){
              f<-sum(pred.vect<tiff.entry)/length(pred.vect)*100
              if(is.na(f)) return(NA)
              if(f==0) return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
              if(0<f & f<=50) return(2*f)
              if(50<=f & f<100) return(2*(100-f))
              if(f==100) return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
              else return(NA)
}

train.dat <- data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
temp <- data.frame(a=runif(130000),b=runif(130000),c=runif(130000),d=runif(130000))
pred.rng<-temp
vnames.final.mod <- names(train.dat)
nvars.final <- length(vnames.final.mod)

start.time<-Sys.time()
             for(k in 1:nvars.final){
                        pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
                        pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
                         }
Sys.time()-start.time    


Rprof("C:\\temp\\mapply.out")
             for(k in 1:nvars.final){
                        pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
                        pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
                         }
Rprof(NULL)
Reply | Threaded
Open this post in threaded view
|

Re: Rprof causing R to crash

Uwe Ligges-3


On 12.12.2012 00:05, Marian Talbert wrote:

> I'm trying to use Rprof() to identify bottlenecks and speed up a particullary
> slow section of code which reads in a portion of a tif file and compares
> each of the values to values of predictors used for model fitting.  I've
> written up an example that anyone can run.  Generally temp would be a
> section of a tif read into a data.frame and used later for other processing.
> The first portion which just records the time works in about 6 seconds the
> second part causes RGui to immediately close with no error or warning.  Any
> advice on how to get Rprof to work or how to speed up this code would be
> greatly appreciated.  I'm using Windows 7 (which might be my problem) and R
> version 2.15.0.

The problem is rather the R version: I cannot reproduce errors with a
recent R.

Uwe Ligges


>
> CalcMESS<-function(tiff.entry,pred.vect){
>                f<-sum(pred.vect<tiff.entry)/length(pred.vect)*100
>                if(is.na(f)) return(NA)
>                if(f==0)
> return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
>                if(0<f & f<=50) return(2*f)
>                if(50<=f & f<100) return(2*(100-f))
>                if(f==100)
> return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
>                else return(NA)
> }
>
> train.dat <- data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
> temp <-
> data.frame(a=runif(130000),b=runif(130000),c=runif(130000),d=runif(130000))
> pred.rng<-temp
> vnames.final.mod <- names(train.dat)
> nvars.final <- length(vnames.final.mod)
>
> start.time<-Sys.time()
>               for(k in 1:nvars.final){
>
> pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
>
> pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
>                           }
> Sys.time()-start.time
>
>
> Rprof("C:\\temp\\mapply.out")
>               for(k in 1:nvars.final){
>
> pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
>
> pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
>                           }
> Rprof(NULL)
>
>
>
> --
> View this message in context: http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Rprof causing R to crash

Jon Skoien
Uwe,

I am unfortunately not able to upgrade to R 2.15.2 right now, but I have
seen a similar problem with several older R versions. If you want to
test with a shorter script, you can try the lines below. These provoke a
crash from a fresh R session on my machine (R 2.15.1 Windows 7):

Rprof()
z = 1
for (i in 1:1e8) z = z+1/i

This runs without problems without the call to Rprof(). I was far from
using all my memory with this call, but you could also check with a
higher number of loops in case you can run these lines and it is somehow
memory related. Just for comparison, I tried 1e9 loops after a call to
Rprof() on one of our Linux servers (R 2.14.0) without any problems.

Jon

On 12-Dec-12 17:06, Uwe Ligges wrote:

>
>
> On 12.12.2012 00:05, Marian Talbert wrote:
>> I'm trying to use Rprof() to identify bottlenecks and speed up a
>> particullary
>> slow section of code which reads in a portion of a tif file and compares
>> each of the values to values of predictors used for model fitting.  I've
>> written up an example that anyone can run.  Generally temp would be a
>> section of a tif read into a data.frame and used later for other
>> processing.
>> The first portion which just records the time works in about 6
>> seconds the
>> second part causes RGui to immediately close with no error or
>> warning.  Any
>> advice on how to get Rprof to work or how to speed up this code would be
>> greatly appreciated.  I'm using Windows 7 (which might be my problem)
>> and R
>> version 2.15.0.
>
> The problem is rather the R version: I cannot reproduce errors with a
> recent R.
>
> Uwe Ligges
>
>
>>
>> CalcMESS<-function(tiff.entry,pred.vect){
>> f<-sum(pred.vect<tiff.entry)/length(pred.vect)*100
>>                if(is.na(f)) return(NA)
>>                if(f==0)
>> return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
>>                if(0<f & f<=50) return(2*f)
>>                if(50<=f & f<100) return(2*(100-f))
>>                if(f==100)
>> return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
>>                else return(NA)
>> }
>>
>> train.dat <-
>> data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
>> temp <-
>> data.frame(a=runif(130000),b=runif(130000),c=runif(130000),d=runif(130000))
>>
>> pred.rng<-temp
>> vnames.final.mod <- names(train.dat)
>> nvars.final <- length(vnames.final.mod)
>>
>> start.time<-Sys.time()
>>               for(k in 1:nvars.final){
>>
>> pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
>>
>> pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
>>
>>                           }
>> Sys.time()-start.time
>>
>>
>> Rprof("C:\\temp\\mapply.out")
>>               for(k in 1:nvars.final){
>>
>> pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
>>
>> pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
>>
>>                           }
>> Rprof(NULL)
>>
>>
>>
>> --
>> View this message in context:
>> http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846.html
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Rprof causing R to crash

Uwe Ligges-3


On 14.12.2012 14:22, Jon Olav Skoien wrote:

> Uwe,
>
> I am unfortunately not able to upgrade to R 2.15.2 right now, but I have
> seen a similar problem with several older R versions. If you want to
> test with a shorter script, you can try the lines below. These provoke a
> crash from a fresh R session on my machine (R 2.15.1 Windows 7):
>
> Rprof()
> z = 1
> for (i in 1:1e8) z = z+1/i
>
> This runs without problems without the call to Rprof(). I was far from
> using all my memory with this call, but you could also check with a
> higher number of loops in case you can run these lines and it is somehow
> memory related. Just for comparison, I tried 1e9 loops after a call to
> Rprof() on one of our Linux servers (R 2.14.0) without any problems.

Works for me.

Uwe


> Jon
>
> On 12-Dec-12 17:06, Uwe Ligges wrote:
>>
>>
>> On 12.12.2012 00:05, Marian Talbert wrote:
>>> I'm trying to use Rprof() to identify bottlenecks and speed up a
>>> particullary
>>> slow section of code which reads in a portion of a tif file and compares
>>> each of the values to values of predictors used for model fitting.  I've
>>> written up an example that anyone can run.  Generally temp would be a
>>> section of a tif read into a data.frame and used later for other
>>> processing.
>>> The first portion which just records the time works in about 6
>>> seconds the
>>> second part causes RGui to immediately close with no error or
>>> warning.  Any
>>> advice on how to get Rprof to work or how to speed up this code would be
>>> greatly appreciated.  I'm using Windows 7 (which might be my problem)
>>> and R
>>> version 2.15.0.
>>
>> The problem is rather the R version: I cannot reproduce errors with a
>> recent R.
>>
>> Uwe Ligges
>>
>>
>>>
>>> CalcMESS<-function(tiff.entry,pred.vect){
>>> f<-sum(pred.vect<tiff.entry)/length(pred.vect)*100
>>>                if(is.na(f)) return(NA)
>>>                if(f==0)
>>> return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
>>>                if(0<f & f<=50) return(2*f)
>>>                if(50<=f & f<100) return(2*(100-f))
>>>                if(f==100)
>>> return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
>>>                else return(NA)
>>> }
>>>
>>> train.dat <-
>>> data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
>>> temp <-
>>> data.frame(a=runif(130000),b=runif(130000),c=runif(130000),d=runif(130000))
>>>
>>> pred.rng<-temp
>>> vnames.final.mod <- names(train.dat)
>>> nvars.final <- length(vnames.final.mod)
>>>
>>> start.time<-Sys.time()
>>>               for(k in 1:nvars.final){
>>>
>>> pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
>>>
>>> pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
>>>
>>>                           }
>>> Sys.time()-start.time
>>>
>>>
>>> Rprof("C:\\temp\\mapply.out")
>>>               for(k in 1:nvars.final){
>>>
>>> pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
>>>
>>> pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
>>>
>>>                           }
>>> Rprof(NULL)
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846.html
>>> Sent from the R help mailing list archive at Nabble.com.
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Rprof causing R to crash

Prof Brian Ripley
In reply to this post by Jon Skoien
On 14/12/2012 13:22, Jon Olav Skoien wrote:
> Uwe,
>
> I am unfortunately not able to upgrade to R 2.15.2 right now, but I have

Why not?   Note that is part of the R-help contract: we only offer any
support for the current version of R (see the posting guide).

The posting guide also asked for 'at a minimum' information: we do not
know (amongst other things) if this is 32- or 64-bit R.  (Your example
works for me in both.)

> seen a similar problem with several older R versions. If you want to
> test with a shorter script, you can try the lines below. These provoke a
> crash from a fresh R session on my machine (R 2.15.1 Windows 7):
>
> Rprof()
> z = 1
> for (i in 1:1e8) z = z+1/i
>
> This runs without problems without the call to Rprof(). I was far from
> using all my memory with this call, but you could also check with a
> higher number of loops in case you can run these lines and it is somehow
> memory related. Just for comparison, I tried 1e9 loops after a call to
> Rprof() on one of our Linux servers (R 2.14.0) without any problems.

Which is irrelevant: a different version of R and a completely different
implementation of Rprof.

>
> Jon
>
> On 12-Dec-12 17:06, Uwe Ligges wrote:
>>
>>
>> On 12.12.2012 00:05, Marian Talbert wrote:
>>> I'm trying to use Rprof() to identify bottlenecks and speed up a
>>> particullary
>>> slow section of code which reads in a portion of a tif file and compares
>>> each of the values to values of predictors used for model fitting.  I've
>>> written up an example that anyone can run.  Generally temp would be a
>>> section of a tif read into a data.frame and used later for other
>>> processing.
>>> The first portion which just records the time works in about 6
>>> seconds the
>>> second part causes RGui to immediately close with no error or
>>> warning.  Any
>>> advice on how to get Rprof to work or how to speed up this code would be
>>> greatly appreciated.  I'm using Windows 7 (which might be my problem)
>>> and R
>>> version 2.15.0.
>>
>> The problem is rather the R version: I cannot reproduce errors with a
>> recent R.
>>
>> Uwe Ligges
>>
>>
>>>
>>> CalcMESS<-function(tiff.entry,pred.vect){
>>> f<-sum(pred.vect<tiff.entry)/length(pred.vect)*100
>>>                if(is.na(f)) return(NA)
>>>                if(f==0)
>>> return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
>>>                if(0<f & f<=50) return(2*f)
>>>                if(50<=f & f<100) return(2*(100-f))
>>>                if(f==100)
>>> return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
>>>                else return(NA)
>>> }
>>>
>>> train.dat <-
>>> data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
>>> temp <-
>>> data.frame(a=runif(130000),b=runif(130000),c=runif(130000),d=runif(130000))
>>>
>>> pred.rng<-temp
>>> vnames.final.mod <- names(train.dat)
>>> nvars.final <- length(vnames.final.mod)
>>>
>>> start.time<-Sys.time()
>>>               for(k in 1:nvars.final){
>>>
>>> pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
>>>
>>> pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
>>>
>>>                           }
>>> Sys.time()-start.time
>>>
>>>
>>> Rprof("C:\\temp\\mapply.out")
>>>               for(k in 1:nvars.final){
>>>
>>> pred.range<-train.dat[,match(vnames.final.mod[k],names(train.dat))]
>>>
>>> pred.rng[,k]<-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
>>>
>>>                           }
>>> Rprof(NULL)
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846.html
>>> Sent from the R help mailing list archive at Nabble.com.
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Rprof causing R to crash

c97sr
Am also having trouble with Rprof.

I am using on R 2.15.2 on the Mac (Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)).

The example above without profiling takes around a minute to run:

> z = 1
> for (i in 1:1e8) z = z+1/i
> z
[1] 19.9979

With profiling:

> Rprof()
> z = 1
> for (i in 1:1e8) z = z+1/i

[INFO] Feb 4, 2013 12:45:20 PM - R stopped.

Crashes within a second or so.

This is consistent with the problem I'm having in my own code in which there is a reasonable amount of memory used (but not a massive amount).

Any help much appreciated. Am happy to post this as a bug if that is now appropriate.

regards

Steven
Reply | Threaded
Open this post in threaded view
|

Re: Rprof causing R to crash

jholtman
Could you be a little more explicit of what the error message is.  On
my Windows system 32-bit, I get the following:

> Rprof()
> z <- 1
> system.time(for (i in 1:1e8) z <- z + 1/i)
Error: cannot allocate vector of size 381.5 Mb

Which is probably due to trying to allocate 1e8 elements of integers
(400MB).  Could you have a similar problem?

Now you could restructure your code to not allocate the large vector by:

Rprof()
z <- 1
i <- 0
system.time(while((i <- i + 1) < 1e8) z <- z + 1/i)
z
Rprof(NULL)

On Mon, Feb 4, 2013 at 7:48 AM, c97sr <[hidden email]> wrote:

> Am also having trouble with Rprof.
>
> I am using on R 2.15.2 on the Mac (Platform: x86_64-apple-darwin9.8.0/x86_64
> (64-bit)).
>
> The example above without profiling takes around a minute to run:
>
>> z = 1
>> for (i in 1:1e8) z = z+1/i
>> z
> [1] 19.9979
>
> With profiling:
>
>> Rprof()
>> z = 1
>> for (i in 1:1e8) z = z+1/i
>
> [INFO] Feb 4, 2013 12:45:20 PM - R stopped.
>
> Crashes within a second or so.
>
> This is consistent with the problem I'm having in my own code in which there
> is a reasonable amount of memory used (but not a massive amount).
>
> Any help much appreciated. Am happy to post this as a bug if that is now
> appropriate.
>
> regards
>
> Steven
>
>
>
> --
> View this message in context: http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846p4657482.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



--
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Rprof causing R to crash

Steven Riley
Hi there,

Thanks for the quick feedback.

There is no further info on the reason for the crash - R just stops.

The for loop is there just to generate the error. I'm following a previous
example that others couldn't reproduce. I need Rprof to work on a different
set of functions that I haven't posted. Those function do involve some
"large" arrays, but have never given any trouble away from Rprof.

I'm pretty sure Rprof is broken. Is there an alternative?

Cheers

Steven

Sent from phone.
On Feb 4, 2013 5:03 PM, "jim holtman" <[hidden email]> wrote:

> Could you be a little more explicit of what the error message is.  On
> my Windows system 32-bit, I get the following:
>
> > Rprof()
> > z <- 1
> > system.time(for (i in 1:1e8) z <- z + 1/i)
> Error: cannot allocate vector of size 381.5 Mb
>
> Which is probably due to trying to allocate 1e8 elements of integers
> (400MB).  Could you have a similar problem?
>
> Now you could restructure your code to not allocate the large vector by:
>
> Rprof()
> z <- 1
> i <- 0
> system.time(while((i <- i + 1) < 1e8) z <- z + 1/i)
> z
> Rprof(NULL)
>
> On Mon, Feb 4, 2013 at 7:48 AM, c97sr <[hidden email]> wrote:
> > Am also having trouble with Rprof.
> >
> > I am using on R 2.15.2 on the Mac (Platform:
> x86_64-apple-darwin9.8.0/x86_64
> > (64-bit)).
> >
> > The example above without profiling takes around a minute to run:
> >
> >> z = 1
> >> for (i in 1:1e8) z = z+1/i
> >> z
> > [1] 19.9979
> >
> > With profiling:
> >
> >> Rprof()
> >> z = 1
> >> for (i in 1:1e8) z = z+1/i
> >
> > [INFO] Feb 4, 2013 12:45:20 PM - R stopped.
> >
> > Crashes within a second or so.
> >
> > This is consistent with the problem I'm having in my own code in which
> there
> > is a reasonable amount of memory used (but not a massive amount).
> >
> > Any help much appreciated. Am happy to post this as a bug if that is now
> > appropriate.
> >
> > regards
> >
> > Steven
> >
> >
> >
> > --
> > View this message in context:
> http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846p4657482.html
> > Sent from the R help mailing list archive at Nabble.com.
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
>
>
> --
> Jim Holtman
> Data Munger Guru
>
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Rprof causing R to crash

c97sr
In reply to this post by jholtman
Hi Jim,

> Could you be a little more explicit of what the error message is.  On
> my Windows system 32-bit, I get the following:

Sorry - it just "stops" as per the message posted. The text is from eclipse.

>  Rprof()
>  z <- 1
>  system.time(for (i in 1:1e8) z <- z + 1/i)
>  Error: cannot allocate vector of size 381.5 Mb

> Which is probably due to trying to allocate 1e8 elements of integers
> (400MB).  Could you have a similar problem?

Shouldn't be. 400MB shouldn't be a problem on a 4GB machine?

Just tried this with no problem on 2.15.0 on "Platform: x86_64-redhat-linux-gnu (64-bit)", so it looks to be particular to my setup on the mac. I also just tried 2.15.2 on the mac using the GUI and  _not_ eclipse, It worked with no problem.

Therefore, it seems that my issues are specific to R 2.15.2 (and R 2.14.0 prior to upgrade) running via StatEt's rj package (with no additional arguments.) I have a workaround now, and I guess this is no longer an issue for the wider group. Will post a link to the StatEt group.

cheers

Steven


Now you could restructure your code to not allocate the large vector by:

Rprof()
z <- 1
i <- 0
system.time(while((i <- i + 1) < 1e8) z <- z + 1/i)
z
Rprof(NULL)

Yes - the example is not related to my code, its just a way of recreating a similar error. I haven't posted my code that causes the error.

Is there a reason that 400 MB should be a problem for RProf? My previous experience is that Rprof is stable with processes with much larger memory footprints. Hence why I think the for loop error might be related to the problems I'm having.

Is there an alternative to Rprof? This is fairly crucial for me right now. Will try to get setup on

cheers

Steven




On Mon, Feb 4, 2013 at 7:48 AM, c97sr <[hidden email]> wrote:
> Am also having trouble with Rprof.
>
> I am using on R 2.15.2 on the Mac (Platform: x86_64-apple-darwin9.8.0/x86_64
> (64-bit)).
>
> The example above without profiling takes around a minute to run:
>
>> z = 1
>> for (i in 1:1e8) z = z+1/i
>> z
> [1] 19.9979
>
> With profiling:
>
>> Rprof()
>> z = 1
>> for (i in 1:1e8) z = z+1/i
>
> [INFO] Feb 4, 2013 12:45:20 PM - R stopped.
>
> Crashes within a second or so.
>
> This is consistent with the problem I'm having in my own code in which there
> is a reasonable amount of memory used (but not a massive amount).
>
> Any help much appreciated. Am happy to post this as a bug if that is now
> appropriate.
>
> regards
>
> Steven
>
>
>
> --
> View this message in context: http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846p4657482.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



--
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.