memory managment under Windows XP

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

memory managment under Windows XP

R user-3
I am using R 2.2.1 in a Windowes XP environment.

I work with very large datasets, and occassionally run
out of memory.

I have modified my boot.ini file to use the "/3gb
switch".

I also run the following line after I launch R ( I am
unsure if it is helpful).

"memory.limit(size = 4095)"

Please point me to useful references on how to better
manage memory, or suggestother actions.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: memory managment under Windows XP

Liaw, Andy
There are plenty in the the list archive, one of which is to switch to a run
64-bit R on a 64-bit platform with lots of physical RAM.  Such hardware is
quite affordable these days (certainly cheaper than most commercial software
that you'd have to buy if you didn't have R).

Andy

From: r user

>
> I am using R 2.2.1 in a Windowes XP environment.
>
> I work with very large datasets, and occassionally run
> out of memory.
>
> I have modified my boot.ini file to use the "/3gb
> switch".
>
> I also run the following line after I launch R ( I am
> unsure if it is helpful).
>
> "memory.limit(size = 4095)"
>
> Please point me to useful references on how to better
> manage memory, or suggestother actions.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: memory managment under Windows XP

Roger J. Bos
And of course using rm(...) to clean up objects you no longer need.  No
amount of physical memory can save you from grossly inefficient code and
large memory leaks.  For example, lets say I have a large testMat object
that I use time period.  I loop though each month using for loops.  Even
though the object has the same name each month, and thus gets overwritten,
the memory management seems to go better by manually removing the object at
the end of each loop.  Also, I sometimes call gc() at the end of each loop,
but I don't know if that actually helps or not.  I figure it can't hurt.





On 2/23/06, Liaw, Andy <[hidden email]> wrote:

>
> There are plenty in the the list archive, one of which is to switch to a
> run
> 64-bit R on a 64-bit platform with lots of physical RAM.  Such hardware is
> quite affordable these days (certainly cheaper than most commercial
> software
> that you'd have to buy if you didn't have R).
>
> Andy
>
> From: r user
> >
> > I am using R 2.2.1 in a Windowes XP environment.
> >
> > I work with very large datasets, and occassionally run
> > out of memory.
> >
> > I have modified my boot.ini file to use the "/3gb
> > switch".
> >
> > I also run the following line after I launch R ( I am
> > unsure if it is helpful).
> >
> > "memory.limit(size = 4095)"
> >
> > Please point me to useful references on how to better
> > manage memory, or suggestother actions.
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide!
> > http://www.R-project.org/posting-guide.html
> >
> >
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: memory managment under Windows XP

Prof Brian Ripley
On Thu, 23 Feb 2006, roger bos wrote:

> And of course using rm(...) to clean up objects you no longer need.  No
> amount of physical memory can save you from grossly inefficient code and
> large memory leaks.  For example, lets say I have a large testMat object
> that I use time period.  I loop though each month using for loops.  Even
> though the object has the same name each month, and thus gets overwritten,
> the memory management seems to go better by manually removing the object at
> the end of each loop.  Also, I sometimes call gc() at the end of each loop,
> but I don't know if that actually helps or not.  I figure it can't hurt.

It does help to rm() and then gc() at a point when you know that the
number of objects in use is minimal.  R will gc() repeatedly when it
starts to run out of address space, but this does not help if the address
space is already fragmented.

The main problem on 32-bit OSes is (virtual) memory fragmentation, and 3Gb
is not really much address space for objects in 100s of Mb.

I've now only got a 64-bit desktop and servers (plus a 32-bit Windows
laptop).  It is a shame for Windows users that a 64-bit Open Source
toolchain* is nowhere in sight, but I suspect a sufficiently determined
user of Win64 could build a 64-bit port of R with commercial compilers.

*You need a compiler, assembler, linker and runtime, and the latter may
well be the most problematic.

> On 2/23/06, Liaw, Andy <[hidden email]> wrote:
>>
>> There are plenty in the the list archive, one of which is to switch to a
>> run
>> 64-bit R on a 64-bit platform with lots of physical RAM.  Such hardware is
>> quite affordable these days (certainly cheaper than most commercial
>> software
>> that you'd have to buy if you didn't have R).
>>
>> Andy
>>
>> From: r user
>>>
>>> I am using R 2.2.1 in a Windowes XP environment.
>>>
>>> I work with very large datasets, and occassionally run
>>> out of memory.
>>>
>>> I have modified my boot.ini file to use the "/3gb
>>> switch".
>>>
>>> I also run the following line after I launch R ( I am
>>> unsure if it is helpful).
>>>
>>> "memory.limit(size = 4095)"
>>>
>>> Please point me to useful references on how to better
>>> manage memory, or suggestother actions.
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide!
>>> http://www.R-project.org/posting-guide.html
>>>
>>>
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide!
>> http://www.R-project.org/posting-guide.html
>>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html