Global counter in library

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Global counter in library

Pascal.Niklaus
I am trying to implement a global counter in a library. This counter
resides in an environment local to that package, and is incremented by
calling a function within this package.

Problems arise when I use parallelization. I protected the increment
operation with a lock so that this operation effectively should be
atomic. However, the environment local to the package seems not to be
shared among the threads that use the functions in that package (not in
the sense that a change would be visible in the other threads).

How is such a global counter best implemented?

Here is a basic sketch of what I tried:


library(parallel)

library(flock)

## begin of code in package

tmp <- tempfile()
.localstuff <- new.env()
.localstuff$N <- 0
.localstuff$tmp <- tempfile()

incr <- function()
{
     h <- lock(.localstuff$tmp)
     .localstuff$N <- .localstuff$N + 1
     unlock(h)
}

## end code in library

invisible(mclapply(1:10, function(i) incr()))

cat("N =", .localstuff$N)

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Global counter in library

Duncan Murdoch-2
On 11/03/2019 2:57 p.m., Pascal A. Niklaus wrote:

> I am trying to implement a global counter in a library. This counter
> resides in an environment local to that package, and is incremented by
> calling a function within this package.
>
> Problems arise when I use parallelization. I protected the increment
> operation with a lock so that this operation effectively should be
> atomic. However, the environment local to the package seems not to be
> shared among the threads that use the functions in that package (not in
> the sense that a change would be visible in the other threads).
>
> How is such a global counter best implemented?

Use the Rdsm package.  See ?Rdsm::Rdsm for help.

Duncan Murdoch

>
> Here is a basic sketch of what I tried:
>
>
> library(parallel)
>
> library(flock)
>
> ## begin of code in package
>
> tmp <- tempfile()
> .localstuff <- new.env()
> .localstuff$N <- 0
> .localstuff$tmp <- tempfile()
>
> incr <- function()
> {
>       h <- lock(.localstuff$tmp)
>       .localstuff$N <- .localstuff$N + 1
>       unlock(h)
> }
>
> ## end code in library
>
> invisible(mclapply(1:10, function(i) incr()))
>
> cat("N =", .localstuff$N)

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Global counter in library

Pascal.Niklaus
Thanks for the hint. I think I will use a big.matrix [with 1 element
only ;-) ] and a mutex to ensure access is thread-safe.

Pascal


On 12.03.19 17:35, Duncan Murdoch wrote:

> On 11/03/2019 2:57 p.m., Pascal A. Niklaus wrote:
>> I am trying to implement a global counter in a library. This counter
>> resides in an environment local to that package, and is incremented by
>> calling a function within this package.
>>
>> Problems arise when I use parallelization. I protected the increment
>> operation with a lock so that this operation effectively should be
>> atomic. However, the environment local to the package seems not to be
>> shared among the threads that use the functions in that package (not in
>> the sense that a change would be visible in the other threads).
>>
>> How is such a global counter best implemented?
>
> Use the Rdsm package.  See ?Rdsm::Rdsm for help.
>
> Duncan Murdoch
>
>>
>> Here is a basic sketch of what I tried:
>>
>>
>> library(parallel)
>>
>> library(flock)
>>
>> ## begin of code in package
>>
>> tmp <- tempfile()
>> .localstuff <- new.env()
>> .localstuff$N <- 0
>> .localstuff$tmp <- tempfile()
>>
>> incr <- function()
>> {
>>       h <- lock(.localstuff$tmp)
>>       .localstuff$N <- .localstuff$N + 1
>>       unlock(h)
>> }
>>
>> ## end code in library
>>
>> invisible(mclapply(1:10, function(i) incr()))
>>
>> cat("N =", .localstuff$N)

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.