R gets slow

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

R gets slow

pau carre
Hello, I have R as a socket server that computes R code sent by some
scripts (the clients). These scrips send R code to generate models
(SVM). The problem is that first models are generated in less than one
second and one hour later, the same models are generated in more than
ten seconds (even training with same data). If I restart the server ,
then it works well (fast). I don't know if I have to free the memory
or something.

Here you have the code:
R server:
FSsocket <- function(){
    continue = TRUE;
    while(continue){
        conn <- try(socketConnection(server = TRUE, port = 7890, blocking =
TRUE, open = "ab"), silent = FALSE);
        isOpened = !inherits(conn, "try-error");
        isOpened = isOpened && isOpen(conn);
        while(!isOpened){
            Sys.sleep(1);
            conn <- try(socketConnection(server = TRUE, port = 7890, blocking
= TRUE, open = "ab"), silent = FALSE);
            isOpened = !inherits(conn, "try-error");
            isOpened = isOpened && isOpen(conn);
        }
        print("Waitting for source");
        srcFile = readLines(conn, n = 1)
        print(srcFile)
        continue = srcFile != "--close"
        if(continue){
            print("Executing source");
            error <- try(source(srcFile), silent = FALSE);
            if(inherits(error, "try-error")){
                writeLines("ERROR", conn)
            }
            print("Sending confirmation")
            writeLines("DONE", conn)
            print("Closing connection")
        }
        close(conn)
    }
}
FSsocket();

Model generator example:
library("class");
library("e1071");
dd = read.table("sampling/adapteddataSet");
attach(dd);
ddv = read.table("sampling/adaptedvalidationDataSet");
attach(ddv);
dd[,1] = factor(dd[,1]);
ddv[,1] = factor(ddv[,1]);
attach(dd);
tr_in = as.matrix(dd[,2:(1 + 1)]);
tr_out = dd[,1];
val_in = as.matrix(ddv[,2:(1 + 1)]);
val_out = ddv[,1];
t = tune(svm, kernel = "radial", train.x = tr_in, train.y = tr_out,
validation.x = val_in, validation.y = val_out,
          range = list( gamma = 2^(-1:1), cost = 2^(2:4) ), tunecontrol =
tune.control(sampling = "fix") )
z = t$best.model
save(z, file = "./models/1/20", compress=FALSE);

Thanks
Pau.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: R gets slow

pau carre
Hello, I am still having the same problem, even removing the objects
(with rm()).
I checked the memory allocated by R and it is constant (about 4.5%),
so I have no idea of what is happening...
Moreover in few minutes (15) the models are generated 5 times
slower... restarting the server is the only (but not acceptable)
solution I have.
(I do not generate any graph and/or extra object)
Pau


2006/3/25, pau carre <[hidden email]>:

> Hello, I have R as a socket server that computes R code sent by some
> scripts (the clients). These scrips send R code to generate models
> (SVM). The problem is that first models are generated in less than one
> second and one hour later, the same models are generated in more than
> ten seconds (even training with same data). If I restart the server ,
> then it works well (fast). I don't know if I have to free the memory
> or something.
>
> Here you have the code:
> R server:
> FSsocket <- function(){
>     continue = TRUE;
>     while(continue){
>         conn <- try(socketConnection(server = TRUE, port = 7890, blocking =
> TRUE, open = "ab"), silent = FALSE);
>         isOpened = !inherits(conn, "try-error");
>         isOpened = isOpened && isOpen(conn);
>         while(!isOpened){
>             Sys.sleep(1);
>             conn <- try(socketConnection(server = TRUE, port = 7890, blocking
> = TRUE, open = "ab"), silent = FALSE);
>             isOpened = !inherits(conn, "try-error");
>             isOpened = isOpened && isOpen(conn);
>         }
>         print("Waitting for source");
>         srcFile = readLines(conn, n = 1)
>         print(srcFile)
>         continue = srcFile != "--close"
>         if(continue){
>             print("Executing source");
>             error <- try(source(srcFile), silent = FALSE);
>             if(inherits(error, "try-error")){
>                 writeLines("ERROR", conn)
>             }
>             print("Sending confirmation")
>             writeLines("DONE", conn)
>             print("Closing connection")
>         }
>         close(conn)
>     }
> }
> FSsocket();
>
> Model generator example:
> library("class");
> library("e1071");
> dd = read.table("sampling/adapteddataSet");
> attach(dd);
> ddv = read.table("sampling/adaptedvalidationDataSet");
> attach(ddv);
> dd[,1] = factor(dd[,1]);
> ddv[,1] = factor(ddv[,1]);
> attach(dd);
> tr_in = as.matrix(dd[,2:(1 + 1)]);
> tr_out = dd[,1];
> val_in = as.matrix(ddv[,2:(1 + 1)]);
> val_out = ddv[,1];
> t = tune(svm, kernel = "radial", train.x = tr_in, train.y = tr_out,
> validation.x = val_in, validation.y = val_out,
>           range = list( gamma = 2^(-1:1), cost = 2^(2:4) ), tunecontrol =
> tune.control(sampling = "fix") )
> z = t$best.model
> save(z, file = "./models/1/20", compress=FALSE);
>
> Thanks
> Pau.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: R gets slow

Liaw, Andy
In reply to this post by pau carre
I'm just taking wild guesses here, and I have no clue what could be the
problem.  

- The timing you mentioned, I assume, is `wall clock' time?  Have you
checked whether the CPU time also increase, or just the wall clock time?

- Perhaps you can use Rprof() to check which part of the computation is
increasing from run to run?

Andy

From: pau carre

>
> Hello, I am still having the same problem, even removing the
> objects (with rm()). I checked the memory allocated by R and
> it is constant (about 4.5%), so I have no idea of what is
> happening... Moreover in few minutes (15) the models are
> generated 5 times slower... restarting the server is the only
> (but not acceptable) solution I have. (I do not generate any
> graph and/or extra object) Pau
>
>
> 2006/3/25, pau carre <[hidden email]>:
> > Hello, I have R as a socket server that computes R code
> sent by some
> > scripts (the clients). These scrips send R code to generate models
> > (SVM). The problem is that first models are generated in
> less than one
> > second and one hour later, the same models are generated in
> more than
> > ten seconds (even training with same data). If I restart
> the server ,
> > then it works well (fast). I don't know if I have to free
> the memory
> > or something.
> >
> > Here you have the code:
> > R server:
> > FSsocket <- function(){
> >     continue = TRUE;
> >     while(continue){
> >         conn <- try(socketConnection(server = TRUE, port = 7890,
> > blocking = TRUE, open = "ab"), silent = FALSE);
> >         isOpened = !inherits(conn, "try-error");
> >         isOpened = isOpened && isOpen(conn);
> >         while(!isOpened){
> >             Sys.sleep(1);
> >             conn <- try(socketConnection(server = TRUE,
> port = 7890,
> > blocking = TRUE, open = "ab"), silent = FALSE);
> >             isOpened = !inherits(conn, "try-error");
> >             isOpened = isOpened && isOpen(conn);
> >         }
> >         print("Waitting for source");
> >         srcFile = readLines(conn, n = 1)
> >         print(srcFile)
> >         continue = srcFile != "--close"
> >         if(continue){
> >             print("Executing source");
> >             error <- try(source(srcFile), silent = FALSE);
> >             if(inherits(error, "try-error")){
> >                 writeLines("ERROR", conn)
> >             }
> >             print("Sending confirmation")
> >             writeLines("DONE", conn)
> >             print("Closing connection")
> >         }
> >         close(conn)
> >     }
> > }
> > FSsocket();
> >
> > Model generator example:
> > library("class");
> > library("e1071");
> > dd = read.table("sampling/adapteddataSet");
> > attach(dd);
> > ddv = read.table("sampling/adaptedvalidationDataSet");
> > attach(ddv);
> > dd[,1] = factor(dd[,1]);
> > ddv[,1] = factor(ddv[,1]);
> > attach(dd);
> > tr_in = as.matrix(dd[,2:(1 + 1)]);
> > tr_out = dd[,1];
> > val_in = as.matrix(ddv[,2:(1 + 1)]);
> > val_out = ddv[,1];
> > t = tune(svm, kernel = "radial", train.x = tr_in, train.y = tr_out,
> > validation.x = val_in, validation.y = val_out,
> >           range = list( gamma = 2^(-1:1), cost = 2^(2:4) ),
> > tunecontrol = tune.control(sampling = "fix") ) z = t$best.model
> > save(z, file = "./models/1/20", compress=FALSE);
> >
> > Thanks
> > Pau.
> >
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: R gets slow

pau carre
In reply to this post by pau carre
Hello, I finally solved the problem. The solution was just remove the
attach operations. When attach is called a copy of the data is
generated and since I know it is not possible to delete it with rm as
it is not associated to any object.

Pau

2006/3/26, pau carre <[hidden email]>:

> Hello, I am still having the same problem, even removing the objects
> (with rm()).
> I checked the memory allocated by R and it is constant (about 4.5%),
> so I have no idea of what is happening...
> Moreover in few minutes (15) the models are generated 5 times
> slower... restarting the server is the only (but not acceptable)
> solution I have.
> (I do not generate any graph and/or extra object)
> Pau
>
>
> 2006/3/25, pau carre <[hidden email]>:
> > Hello, I have R as a socket server that computes R code sent by some
> > scripts (the clients). These scrips send R code to generate models
> > (SVM). The problem is that first models are generated in less than one
> > second and one hour later, the same models are generated in more than
> > ten seconds (even training with same data). If I restart the server ,
> > then it works well (fast). I don't know if I have to free the memory
> > or something.
> >
> > Here you have the code:
> > R server:
> > FSsocket <- function(){
> >     continue = TRUE;
> >     while(continue){
> >         conn <- try(socketConnection(server = TRUE, port = 7890, blocking =
> > TRUE, open = "ab"), silent = FALSE);
> >         isOpened = !inherits(conn, "try-error");
> >         isOpened = isOpened && isOpen(conn);
> >         while(!isOpened){
> >             Sys.sleep(1);
> >             conn <- try(socketConnection(server = TRUE, port = 7890, blocking
> > = TRUE, open = "ab"), silent = FALSE);
> >             isOpened = !inherits(conn, "try-error");
> >             isOpened = isOpened && isOpen(conn);
> >         }
> >         print("Waitting for source");
> >         srcFile = readLines(conn, n = 1)
> >         print(srcFile)
> >         continue = srcFile != "--close"
> >         if(continue){
> >             print("Executing source");
> >             error <- try(source(srcFile), silent = FALSE);
> >             if(inherits(error, "try-error")){
> >                 writeLines("ERROR", conn)
> >             }
> >             print("Sending confirmation")
> >             writeLines("DONE", conn)
> >             print("Closing connection")
> >         }
> >         close(conn)
> >     }
> > }
> > FSsocket();
> >
> > Model generator example:
> > library("class");
> > library("e1071");
> > dd = read.table("sampling/adapteddataSet");
> > attach(dd);
> > ddv = read.table("sampling/adaptedvalidationDataSet");
> > attach(ddv);
> > dd[,1] = factor(dd[,1]);
> > ddv[,1] = factor(ddv[,1]);
> > attach(dd);
> > tr_in = as.matrix(dd[,2:(1 + 1)]);
> > tr_out = dd[,1];
> > val_in = as.matrix(ddv[,2:(1 + 1)]);
> > val_out = ddv[,1];
> > t = tune(svm, kernel = "radial", train.x = tr_in, train.y = tr_out,
> > validation.x = val_in, validation.y = val_out,
> >           range = list( gamma = 2^(-1:1), cost = 2^(2:4) ), tunecontrol =
> > tune.control(sampling = "fix") )
> > z = t$best.model
> > save(z, file = "./models/1/20", compress=FALSE);
> >
> > Thanks
> > Pau.
> >
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: R gets slow

Peter Dalgaard
"pau carre" <[hidden email]> writes:

> Hello, I finally solved the problem. The solution was just remove the
> attach operations. When attach is called a copy of the data is
> generated and since I know it is not possible to delete it with rm as
> it is not associated to any object.

You can detach() it though...
 

> Pau
>
> 2006/3/26, pau carre <[hidden email]>:
> > Hello, I am still having the same problem, even removing the objects
> > (with rm()).
> > I checked the memory allocated by R and it is constant (about 4.5%),
> > so I have no idea of what is happening...
> > Moreover in few minutes (15) the models are generated 5 times
> > slower... restarting the server is the only (but not acceptable)
> > solution I have.
> > (I do not generate any graph and/or extra object)
> > Pau
> >
> >
> > 2006/3/25, pau carre <[hidden email]>:
> > > Hello, I have R as a socket server that computes R code sent by some
> > > scripts (the clients). These scrips send R code to generate models
> > > (SVM). The problem is that first models are generated in less than one
> > > second and one hour later, the same models are generated in more than
> > > ten seconds (even training with same data). If I restart the server ,
> > > then it works well (fast). I don't know if I have to free the memory
> > > or something.
> > >
> > > Here you have the code:
> > > R server:
> > > FSsocket <- function(){
> > >     continue = TRUE;
> > >     while(continue){
> > >         conn <- try(socketConnection(server = TRUE, port = 7890, blocking =
> > > TRUE, open = "ab"), silent = FALSE);
> > >         isOpened = !inherits(conn, "try-error");
> > >         isOpened = isOpened && isOpen(conn);
> > >         while(!isOpened){
> > >             Sys.sleep(1);
> > >             conn <- try(socketConnection(server = TRUE, port = 7890, blocking
> > > = TRUE, open = "ab"), silent = FALSE);
> > >             isOpened = !inherits(conn, "try-error");
> > >             isOpened = isOpened && isOpen(conn);
> > >         }
> > >         print("Waitting for source");
> > >         srcFile = readLines(conn, n = 1)
> > >         print(srcFile)
> > >         continue = srcFile != "--close"
> > >         if(continue){
> > >             print("Executing source");
> > >             error <- try(source(srcFile), silent = FALSE);
> > >             if(inherits(error, "try-error")){
> > >                 writeLines("ERROR", conn)
> > >             }
> > >             print("Sending confirmation")
> > >             writeLines("DONE", conn)
> > >             print("Closing connection")
> > >         }
> > >         close(conn)
> > >     }
> > > }
> > > FSsocket();
> > >
> > > Model generator example:
> > > library("class");
> > > library("e1071");
> > > dd = read.table("sampling/adapteddataSet");
> > > attach(dd);
> > > ddv = read.table("sampling/adaptedvalidationDataSet");
> > > attach(ddv);
> > > dd[,1] = factor(dd[,1]);
> > > ddv[,1] = factor(ddv[,1]);
> > > attach(dd);
> > > tr_in = as.matrix(dd[,2:(1 + 1)]);
> > > tr_out = dd[,1];
> > > val_in = as.matrix(ddv[,2:(1 + 1)]);
> > > val_out = ddv[,1];
> > > t = tune(svm, kernel = "radial", train.x = tr_in, train.y = tr_out,
> > > validation.x = val_in, validation.y = val_out,
> > >           range = list( gamma = 2^(-1:1), cost = 2^(2:4) ), tunecontrol =
> > > tune.control(sampling = "fix") )
> > > z = t$best.model
> > > save(z, file = "./models/1/20", compress=FALSE);
> > >
> > > Thanks
> > > Pau.
> > >
> >
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>

--
   O__  ---- Peter Dalgaard             Ă˜ster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark          Ph:  (+45) 35327918
~~~~~~~~~~ - ([hidden email])                  FAX: (+45) 35327907

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html