Package gamlss used inside foreach() and %dopar% fails to find an object

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Package gamlss used inside foreach() and %dopar% fails to find an object

Nik Tuzov

Hello all:

Please help me with this "can't find object" issue. I'm trying to get leave-one-out predicted values for Beta-binomial regression.  
It may be the gamlss issue because the code seems to work when %do% is used. I have searched for similar issues, but haven't managed to figure it out. This is on Windows 10 platform.

Thanks in advance,
Nik

# --------------------------------------------------------------

library('gamlss')
library('foreach')
library('doParallel')

registerDoParallel(cores = 4)
# Generate data
set.seed(314)
sample.size <- 30
input.processed.cut <- data.frame(TP = round(runif(sample.size) * 100),
                                  FP = round(runif(sample.size) * 100),
                                  x = runif(sample.size))
# Fit Beta-binomial
model3 <- gamlss(formula = cbind(TP, FP) ~ x,  
                 family = BB,  
                 data = input.processed.cut)

# Get the leave-one-out values
loo_predict.mu <- function(model.obj, input.data) {
  yhat <- foreach(i = 1 : nrow(input.data), .packages="gamlss", .combine = rbind) %dopar% {
    updated.model.obj <- update(model.obj, data = input.data[-i, ])
    predict(updated.model.obj, what = "mu", newdata = input.data[i,], type = "response")
  }
  return(data.frame(result = yhat[, 1], row.names = NULL))
}

par.run <- loo_predict.mu(model3, input.processed.cut)

# Error in { : task 1 failed - "object 'input.data' not found"

# ------------------------------------------------------------------------

> version
               _                          
platform       x86_64-w64-mingw32          
arch           x86_64                      
os             mingw32                    
system         x86_64, mingw32            
status                                    
major          3                          
minor          4.3                        
year           2017                        
month          11                          
day            30                          
svn rev        73796                      
language       R                          
version.string R version 3.4.3 (2017-11-30)
nickname       Kite-Eating Tree

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Package gamlss used inside foreach() and %dopar% fails to find an object

Jeff Newmiller
If the code you are running in parallel is complicated, maybe foreach is not sophisticated enough to find all the variables you refer to. Maybe use parallel::clusterExport yourself? But be a aware that passing parameters is much safer than directly accessing globals in parallel processing, so this might just be your warning to not do that anyway.
--
Sent from my phone. Please excuse my brevity.

On March 9, 2018 7:50:44 AM PST, Nik Tuzov <[hidden email]> wrote:

>
>Hello all:
>
>Please help me with this "can't find object" issue. I'm trying to get
>leave-one-out predicted values for Beta-binomial regression.  
>It may be the gamlss issue because the code seems to work when %do% is
>used. I have searched for similar issues, but haven't managed to figure
>it out. This is on Windows 10 platform.
>
>Thanks in advance,
>Nik
>
># --------------------------------------------------------------
>
>library('gamlss')
>library('foreach')
>library('doParallel')
>
>registerDoParallel(cores = 4)
># Generate data
>set.seed(314)
>sample.size <- 30
>input.processed.cut <- data.frame(TP = round(runif(sample.size) * 100),
>
>                                 FP = round(runif(sample.size) * 100),
>                                  x = runif(sample.size))
># Fit Beta-binomial
>model3 <- gamlss(formula = cbind(TP, FP) ~ x,  
>                 family = BB,  
>                 data = input.processed.cut)
>
># Get the leave-one-out values
>loo_predict.mu <- function(model.obj, input.data) {
>yhat <- foreach(i = 1 : nrow(input.data), .packages="gamlss", .combine
>= rbind) %dopar% {
>    updated.model.obj <- update(model.obj, data = input.data[-i, ])
>predict(updated.model.obj, what = "mu", newdata = input.data[i,], type
>= "response")
>  }
>  return(data.frame(result = yhat[, 1], row.names = NULL))
>}
>
>par.run <- loo_predict.mu(model3, input.processed.cut)
>
># Error in { : task 1 failed - "object 'input.data' not found"
>
>#
>------------------------------------------------------------------------
>
>> version
>               _                          
>platform       x86_64-w64-mingw32          
>arch           x86_64                      
>os             mingw32                    
>system         x86_64, mingw32            
>status                                    
>major          3                          
>minor          4.3                        
>year           2017                        
>month          11                          
>day            30                          
>svn rev        73796                      
>language       R                          
>version.string R version 3.4.3 (2017-11-30)
>nickname       Kite-Eating Tree
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Package gamlss used inside foreach() and %dopar% fails to find an object

Nik Tuzov

Hello Jeff:

Thanks for replying.
I made a mistake in my original post.
The same error is generated with %do% as well, so it seems to be gamlss-related.

Nik


----- Original Message -----
From: "Jeff Newmiller" <[hidden email]>
To: "r-help" <[hidden email]>, "Nik Tuzov" <[hidden email]>
Sent: Friday, March 9, 2018 11:16:21 AM
Subject: Re: [R] Package gamlss used inside foreach() and %dopar% fails to find an object

If the code you are running in parallel is complicated, maybe foreach is not sophisticated enough to find all the variables you refer to. Maybe use parallel::clusterExport yourself? But be a aware that passing parameters is much safer than directly accessing globals in parallel processing, so this might just be your warning to not do that anyway.
--
Sent from my phone. Please excuse my brevity.

On March 9, 2018 7:50:44 AM PST, Nik Tuzov <[hidden email]> wrote:

>
>Hello all:
>
>Please help me with this "can't find object" issue. I'm trying to get
>leave-one-out predicted values for Beta-binomial regression.  
>It may be the gamlss issue because the code seems to work when %do% is
>used. I have searched for similar issues, but haven't managed to figure
>it out. This is on Windows 10 platform.
>
>Thanks in advance,
>Nik
>
># --------------------------------------------------------------
>
>library('gamlss')
>library('foreach')
>library('doParallel')
>
>registerDoParallel(cores = 4)
># Generate data
>set.seed(314)
>sample.size <- 30
>input.processed.cut <- data.frame(TP = round(runif(sample.size) * 100),
>
>                                 FP = round(runif(sample.size) * 100),
>                                  x = runif(sample.size))
># Fit Beta-binomial
>model3 <- gamlss(formula = cbind(TP, FP) ~ x,  
>                 family = BB,  
>                 data = input.processed.cut)
>
># Get the leave-one-out values
>loo_predict.mu <- function(model.obj, input.data) {
>yhat <- foreach(i = 1 : nrow(input.data), .packages="gamlss", .combine
>= rbind) %dopar% {
>    updated.model.obj <- update(model.obj, data = input.data[-i, ])
>predict(updated.model.obj, what = "mu", newdata = input.data[i,], type
>= "response")
>  }
>  return(data.frame(result = yhat[, 1], row.names = NULL))
>}
>
>par.run <- loo_predict.mu(model3, input.processed.cut)
>
># Error in { : task 1 failed - "object 'input.data' not found"
>
>#
>------------------------------------------------------------------------
>
>> version
>               _                          
>platform       x86_64-w64-mingw32          
>arch           x86_64                      
>os             mingw32                    
>system         x86_64, mingw32            
>status                                    
>major          3                          
>minor          4.3                        
>year           2017                        
>month          11                          
>day            30                          
>svn rev        73796                      
>language       R                          
>version.string R version 3.4.3 (2017-11-30)
>nickname       Kite-Eating Tree
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.