Snow on Windows Cluster

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Snow on Windows Cluster

Markus Schmidberger
Hello,

the package snow is not working on a windows cluster with MPICH2 and
Rmpi. There is an error in makeCluster:

launch failed: CreateProcess(/usr/bin/env
"RPROG="C:\Programme\R\R-2.5.1\bin\R" "OUT=/dev/null" "R_LIBS="
C:/Programme/R/R-2.5.1/library/snow/RMPInode.sh) on 'cl1' failed, error
3 - Das System kann den angegbenen Pfad nicht finden.

I looked into makeMPIcluster. It could not work, mpi.comm.spawn gets the
shell command for unix to start the slaves. A simple solution is to use
the windows-code from mpi.spawn.rslaves, then it looks like this. And is
working on our windows-cluster. (small failures on stopping cluster)

# Fro windows
if (.Platform$OS=="windows"){
    workdrive <- unlist(strsplit(getwd(),":"))[1]
    tmpdrive <- unlist(strsplit(tempdir(),":"))[1]
    worktmp <- as.logical(toupper(workdrive)==toupper(tmpdrive))
    tmpdir <- unlist(strsplit(tempdir(),"/Rtmp"))[1]
    localhost <- Sys.getenv("COMPUTERNAME")
    networkdrive <-.Call("RegQuery",
as.integer(2),paste("NETWORK\\",workdrive,sep=""),
                       PACKAGE="Rmpi")
    remotepath <-networkdrive[which(networkdrive=="RemotePath")+1]
    mapdrive=TRUE
    mapdrive <- as.logical(mapdrive && !is.null(remotepath))
    Rscript=system.file("slaveload.R", package="snow")
    arg <- c(Rscript, R.home(), workdrive, getwd(),worktmp, tmpdir,
                   localhost, mapdrive, remotepath)
           
#Rscript,R_HOME,WDrive,WDir,WorkTmp,TmpDir,Master,MapDrive,RemotePath
    print("super")
    count<-mpi.comm.spawn(
        slave=system.file("Rslaves.bat", package="Rmpi"),
        slavearg=arg,
        nslaves=count)
} else{   # for unix
    count <- mpi.comm.spawn(slave = "/usr/bin/env",
                                slavearg = args,
                                nslaves = count,
                                intercomm = intercomm)
}

Doing some work, there should be a better and nicer solution with
directly using mpi.spawn.Rslaves.

Best
Markus

--
Dipl.-Tech. Math. Markus Schmidberger

Ludwig-Maximilians-Universität München
IBE - Institut für medizinische Informationsverarbeitung,
Biometrie und Epidemiologie
Marchioninistr. 15, D-81377 Muenchen
URL: http://ibe.web.med.uni-muenchen.de 
Mail: Markus.Schmidberger [at] ibe.med.uni-muenchen.de

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Snow on Windows Cluster

Luke Tierney
Thanks for the note.

Could you give me more info on the "small failures on stopping cluster"?

I don't currently have a windows system I can test this on so I can't
fold it into the snow release yet. But hopefully I can in the next
month or so. I may get back in touch at that point.

Meanwhile I have been working with a modified shell script that can be
used with mpirun under lam on unix.  I've attached the current
version.  It may turn out that this approach is a little more
maitainable on the Windows side as well.  The way this gets used is

     mpirun -np 5 RMPISNOW

to run a master and 4 workers, and then in the master

     cl <- getMPIcluster()

instead of using makeCluster or makeMPIcluster.  I have not yet found
the MICH2 analog of the LAMRANK environment variable but suspect
something similar does exist.

Best,

luke


On Mon, 3 Sep 2007, Markus Schmidberger wrote:

> Hello,
>
> the package snow is not working on a windows cluster with MPICH2 and Rmpi.
> There is an error in makeCluster:
>
> launch failed: CreateProcess(/usr/bin/env
> "RPROG="C:\Programme\R\R-2.5.1\bin\R" "OUT=/dev/null" "R_LIBS="
> C:/Programme/R/R-2.5.1/library/snow/RMPInode.sh) on 'cl1' failed, error 3 -
> Das System kann den angegbenen Pfad nicht finden.
>
> I looked into makeMPIcluster. It could not work, mpi.comm.spawn gets the
> shell command for unix to start the slaves. A simple solution is to use the
> windows-code from mpi.spawn.rslaves, then it looks like this. And is working
> on our windows-cluster. (small failures on stopping cluster)
>
> # Fro windows
> if (.Platform$OS=="windows"){
>   workdrive <- unlist(strsplit(getwd(),":"))[1]
>   tmpdrive <- unlist(strsplit(tempdir(),":"))[1]
>   worktmp <- as.logical(toupper(workdrive)==toupper(tmpdrive))
>   tmpdir <- unlist(strsplit(tempdir(),"/Rtmp"))[1]
>   localhost <- Sys.getenv("COMPUTERNAME")
>   networkdrive <-.Call("RegQuery",
> as.integer(2),paste("NETWORK\\",workdrive,sep=""),
>                      PACKAGE="Rmpi")
>   remotepath <-networkdrive[which(networkdrive=="RemotePath")+1]
>   mapdrive=TRUE
>   mapdrive <- as.logical(mapdrive && !is.null(remotepath))
>   Rscript=system.file("slaveload.R", package="snow")
>   arg <- c(Rscript, R.home(), workdrive, getwd(),worktmp, tmpdir,
>                  localhost, mapdrive, remotepath)
>           #Rscript,R_HOME,WDrive,WDir,WorkTmp,TmpDir,Master,MapDrive,RemotePath
>   print("super")
>   count<-mpi.comm.spawn(
>       slave=system.file("Rslaves.bat", package="Rmpi"),
>       slavearg=arg,
>       nslaves=count)
> } else{   # for unix
>   count <- mpi.comm.spawn(slave = "/usr/bin/env",
>                               slavearg = args,
>                               nslaves = count,
>                               intercomm = intercomm)
> }
>
> Doing some work, there should be a better and nicer solution with directly
> using mpi.spawn.Rslaves.
>
> Best
> Markus
>
>
--
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:      [hidden email]
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

RMPISNOW (1K) Download Attachment