V2.9.0 changes [Sec=Unclassified]

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

V2.9.0 changes [Sec=Unclassified]

Troy Robertson
Hi all,



Prefix: I am a frustrated Java coder in R.



I am coding a medium sized ecosystem modelling program in R.  I have changed to using S4 objects and it has cost me an order of magnitude in execution speed over the functional model.  I cannot afford this penalty and have found that it is the result of all the passing-by-value of objects.



I see that you can now safely inherit from environment in V2.9.0.

That got me all excited that I would now be able to pass objects by reference.



But...

That doesn't seem to be the case.

It only seem that passing an environment which holds the object allows for pass-by-reference and that passing an object which inherits from environment doesn't.

Why is this the case, either an object inherits the properties of its parent or it doesn't.

Has anyone else had a play with this?  Or have I got it all wrong.



I tried the below:

-----------------------------------------------------------------------------------------

setClass('foo', representation=representation(stuff='list', bar='numeric'),

                     prototype=list(stuff=list(), bar=0),

                     contains='.environment')



setGeneric('doit', function(.Object, newfoo='environment') standardGeneric('doit'))



setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})



z <- new('foo')



z@stuff$x <- new('foo')



doit(z,z@stuff$x)



z@stuff$x@bar



[1] 0

------------------------------------------------------------------------------------------



Can anyone help with a better way of doing this.

I'm trying to avoid all the indirection of packing and unpacking environments for passing.





Thanks heaps



Troy





Troy Robertson

Database and Computing Support Provider

Southern Ocean Ecosystems, ERM/Fish

Australian Antarctic Division

Channel Highway, Kingston 7050

PH: 03 62323571

[hidden email]




___________________________________________________________________________

    Australian Antarctic Division - Commonwealth of Australia
IMPORTANT: This transmission is intended for the addressee only. If you are not the
intended recipient, you are notified that use or dissemination of this communication is
strictly prohibited by Commonwealth law. If you have received this transmission in error,
please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
DELETE the message.
        Visit our web site at http://www.antarctica.gov.au/
___________________________________________________________________________

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Antonio, Fabio Di Narzo
Not a direct answer to your question, but...
You might consider using the R.oo package, from H. Bengtsson. It's
very stable, written in pure R, and cleanly allows you to do
pass-by-reference OO programming, with no tricks.

HTH,
af

2009/6/23 Troy Robertson <[hidden email]>:

> Hi all,
>
>
>
> Prefix: I am a frustrated Java coder in R.
>
>
>
> I am coding a medium sized ecosystem modelling program in R.  I have changed to using S4 objects and it has cost me an order of magnitude in execution speed over the functional model.  I cannot afford this penalty and have found that it is the result of all the passing-by-value of objects.
>
>
>
> I see that you can now safely inherit from environment in V2.9.0.
>
> That got me all excited that I would now be able to pass objects by reference.
>
>
>
> But...
>
> That doesn't seem to be the case.
>
> It only seem that passing an environment which holds the object allows for pass-by-reference and that passing an object which inherits from environment doesn't.
>
> Why is this the case, either an object inherits the properties of its parent or it doesn't.
>
> Has anyone else had a play with this?  Or have I got it all wrong.
>
>
>
> I tried the below:
>
> -----------------------------------------------------------------------------------------
>
> setClass('foo', representation=representation(stuff='list', bar='numeric'),
>
>                     prototype=list(stuff=list(), bar=0),
>
>                     contains='.environment')
>
>
>
> setGeneric('doit', function(.Object, newfoo='environment') standardGeneric('doit'))
>
>
>
> setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
>
>
>
> z <- new('foo')
>
>
>
> z@stuff$x <- new('foo')
>
>
>
> doit(z,z@stuff$x)
>
>
>
> z@stuff$x@bar
>
>
>
> [1] 0
>
> ------------------------------------------------------------------------------------------
>
>
>
> Can anyone help with a better way of doing this.
>
> I'm trying to avoid all the indirection of packing and unpacking environments for passing.
>
>
>
>
>
> Thanks heaps
>
>
>
> Troy
>
>
>
>
>
> Troy Robertson
>
> Database and Computing Support Provider
>
> Southern Ocean Ecosystems, ERM/Fish
>
> Australian Antarctic Division
>
> Channel Highway, Kingston 7050
>
> PH: 03 62323571
>
> [hidden email]
>
>
>
>
> ___________________________________________________________________________
>
>    Australian Antarctic Division - Commonwealth of Australia
> IMPORTANT: This transmission is intended for the addressee only. If you are not the
> intended recipient, you are notified that use or dissemination of this communication is
> strictly prohibited by Commonwealth law. If you have received this transmission in error,
> please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
> DELETE the message.
>        Visit our web site at http://www.antarctica.gov.au/
> ___________________________________________________________________________
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>



--
Antonio, Fabio Di Narzo
Ph.D. student at
Department of Statistical Sciences
University of Bologna, Italy

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Martin Morgan
In reply to this post by Troy Robertson
Troy Robertson wrote:
> Hi all,
>
>
>
> Prefix: I am a frustrated Java coder in R.

ah good, if you're frustrated with Java you'll find R very different ;)

>
>
>
> I am coding a medium sized ecosystem modelling program in R.  I have changed to using S4 objects and it has cost me an order of magnitude in execution speed over the functional model.  I cannot afford this penalty and have found that it is the result of all the passing-by-value of objects.
>
>
>
> I see that you can now safely inherit from environment in V2.9.0.
>
> That got me all excited that I would now be able to pass objects by reference.
>
>
>
> But...
>
> That doesn't seem to be the case.
>
> It only seem that passing an environment which holds the object allows for pass-by-reference and that passing an object which inherits from environment doesn't.
>
> Why is this the case, either an object inherits the properties of its parent or it doesn't.

The object inherits slots from it's parent, and the methods defined on
the parent class. Maybe this example helps?

setClass("Ticker", contains=".environment")

## initialize essential, so each instance gets its own environment
setMethod(initialize, "Ticker",
          function(.Object, ..., .xData=new.env(parent=emptyenv()))
{
    .xData[["count"]] <- 0
    callNextMethod(.Object, ..., .xData=.xData)
})

## tick: increment (private) counter by n
setGeneric("tick", function(reference, n=1L) standardGeneric("tick"),
           signature="reference")

setMethod(tick, "Ticker", function(reference, n=1L) {
    reference[["count"]] <- reference[["count"]] + n
})

## tock: report current value of counter
setGeneric("tock", function(reference) standardGeneric("tock"))

setMethod(tock, "Ticker", function(reference) {
    reference[["count"]]
})

and then

> e <- new("Ticker")
> tock(e)
[1] 0
> tick(e); tick(e, 10); tock(e)
[1] 11
> f <- e
> tock(f); tick(e); tock(f)
[1] 11
[1] 12

The data inside .environment could be structured, too, using S4.
Probably it would be more appropriate to have the environment as a slot,
rather the class that is being extended. And in terms of inherited
'properties', e.g., the "[[" function as defined on environments is
available

> e[["count"]]

Of course I haven't seen your code, but a different interpretation of
your performance issues is that, within the rules of S4, you've chosen
to implement functionality in an inefficient way. So it might be
instructive to step back a bit and try to reformulate your data
structures and methods. This is hard to do.

Martin

>
> Has anyone else had a play with this?  Or have I got it all wrong.
>
>
>
> I tried the below:
>
> -----------------------------------------------------------------------------------------
>
> setClass('foo', representation=representation(stuff='list', bar='numeric'),
>
>                      prototype=list(stuff=list(), bar=0),
>
>                      contains='.environment')
>
>
>
> setGeneric('doit', function(.Object, newfoo='environment') standardGeneric('doit'))
>
>
>
> setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
>
>
>
> z <- new('foo')
>
>
>
> z@stuff$x <- new('foo')
>
>
>
> doit(z,z@stuff$x)
>
>
>
> z@stuff$x@bar
>
>
>
> [1] 0
>
> ------------------------------------------------------------------------------------------
>
>
>
> Can anyone help with a better way of doing this.
>
> I'm trying to avoid all the indirection of packing and unpacking environments for passing.
>
>
>
>
>
> Thanks heaps
>
>
>
> Troy
>
>
>
>
>
> Troy Robertson
>
> Database and Computing Support Provider
>
> Southern Ocean Ecosystems, ERM/Fish
>
> Australian Antarctic Division
>
> Channel Highway, Kingston 7050
>
> PH: 03 62323571
>
> [hidden email]
>
>
>
>
> ___________________________________________________________________________
>
>     Australian Antarctic Division - Commonwealth of Australia
> IMPORTANT: This transmission is intended for the addressee only. If you are not the
> intended recipient, you are notified that use or dissemination of this communication is
> strictly prohibited by Commonwealth law. If you have received this transmission in error,
> please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
> DELETE the message.
>         Visit our web site at http://www.antarctica.gov.au/
> ___________________________________________________________________________
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Troy Robertson
In reply to this post by Antonio, Fabio Di Narzo
Yes, I had looked at R.oo, S4 and proto before beginning coding. I had initially assumed that S4 was an enhancement of or replacement to R.oo that was implemented at a lower level and had decided to go with the 'future' of OO in R.

These assumptions were not necessarily correct.

Troy

Troy Robertson
Database and Computing Support Provider
Southern Ocean Ecosystems, ERM/Fish
Australian Antarctic Division
Channel Highway, Kingston 7050
PH: 03 62323571
[hidden email]


> -----Original Message-----
> From: Antonio, Fabio Di Narzo [mailto:[hidden email]]
> Sent: Tuesday, 23 June 2009 6:22 PM
> To: Troy Robertson
> Cc: [hidden email]
> Subject: Re: [Rd] V2.9.0 changes [Sec=Unclassified]
>
> Not a direct answer to your question, but...
> You might consider using the R.oo package, from H. Bengtsson. It's
> very stable, written in pure R, and cleanly allows you to do
> pass-by-reference OO programming, with no tricks.
>
> HTH,
> af
>
> 2009/6/23 Troy Robertson <[hidden email]>:
> > Hi all,
> >
> >
> >
> > Prefix: I am a frustrated Java coder in R.
> >
> >
> >
> > I am coding a medium sized ecosystem modelling program in R.  I have
> changed to using S4 objects and it has cost me an order of magnitude in
> execution speed over the functional model.  I cannot afford this penalty
> and have found that it is the result of all the passing-by-value of
> objects.
> >
> >
> >
> > I see that you can now safely inherit from environment in V2.9.0.
> >
> > That got me all excited that I would now be able to pass objects by
> reference.
> >
> >
> >
> > But...
> >
> > That doesn't seem to be the case.
> >
> > It only seem that passing an environment which holds the object allows
> for pass-by-reference and that passing an object which inherits from
> environment doesn't.
> >
> > Why is this the case, either an object inherits the properties of its
> parent or it doesn't.
> >
> > Has anyone else had a play with this?  Or have I got it all wrong.
> >
> >
> >
> > I tried the below:
> >
> > ------------------------------------------------------------------------
> -----------------
> >
> > setClass('foo', representation=representation(stuff='list',
> bar='numeric'),
> >
> >                     prototype=list(stuff=list(), bar=0),
> >
> >                     contains='.environment')
> >
> >
> >
> > setGeneric('doit', function(.Object, newfoo='environment')
> standardGeneric('doit'))
> >
> >
> >
> > setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
> >
> >
> >
> > z <- new('foo')
> >
> >
> >
> > z@stuff$x <- new('foo')
> >
> >
> >
> > doit(z,z@stuff$x)
> >
> >
> >
> > z@stuff$x@bar
> >
> >
> >
> > [1] 0
> >
> > ------------------------------------------------------------------------
> ------------------
> >
> >
> >
> > Can anyone help with a better way of doing this.
> >
> > I'm trying to avoid all the indirection of packing and unpacking
> environments for passing.
> >
> >
> >
> >
> >
> > Thanks heaps
> >
> >
> >
> > Troy
> >
> >
> >
> >
> >
> > Troy Robertson
> >
> > Database and Computing Support Provider
> >
> > Southern Ocean Ecosystems, ERM/Fish
> >
> > Australian Antarctic Division
> >
> > Channel Highway, Kingston 7050
> >
> > PH: 03 62323571
> >
> > [hidden email]
> >
> >
> >
> >
> >
> __________________________________________________________________________
> _
> >
> >    Australian Antarctic Division - Commonwealth of Australia
> > IMPORTANT: This transmission is intended for the addressee only. If you
> are not the
> > intended recipient, you are notified that use or dissemination of this
> communication is
> > strictly prohibited by Commonwealth law. If you have received this
> transmission in error,
> > please notify the sender immediately by e-mail or by telephoning +61 3
> 6232 3209 and
> > DELETE the message.
> >        Visit our web site at http://www.antarctica.gov.au/
> >
> __________________________________________________________________________
> _
> >
> >        [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
>
>
> --
> Antonio, Fabio Di Narzo
> Ph.D. student at
> Department of Statistical Sciences
> University of Bologna, Italy
___________________________________________________________________________

    Australian Antarctic Division - Commonwealth of Australia
IMPORTANT: This transmission is intended for the addressee only. If you are not the
intended recipient, you are notified that use or dissemination of this communication is
strictly prohibited by Commonwealth law. If you have received this transmission in error,
please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
DELETE the message.
        Visit our web site at http://www.antarctica.gov.au/
___________________________________________________________________________

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Troy Robertson
In reply to this post by Martin Morgan
Thanks Martin,

Yes I can see that a slot can be initialised as an environment, and data stored within that environment but it seems to then require an additional layer of indirection to access data members within the environment and negates the use of slots in the first place.

The same goes for the "[[" function which works on an object which extends environment as I would expect slots within that object to work (but don't), eg allowing a pass-by-reference arrangement.

I hadn't encountered the use of .xdata as you demonstrated below and think I can work with this way of assigning data members to an S4 object, allowing for a pass-by-reference mechanism.  It just seems to negate the point of providing a slot mechanism for objects.

As for stepping back and re-thinking the way I am implementing my data structures and methods.  It is difficult coming from one way of coding in a different language.  It's easy to try and enforce that way of doing things on the new language.  Can you point me to any examples of general programming using S4 that I could use to change my way of thinking about the problem?

I will have a play around more tomorrow.

Cheers

Troy

> -----Original Message-----
> From: Martin Morgan [mailto:[hidden email]]
> Sent: Tuesday, 23 June 2009 11:25 PM
> To: Troy Robertson
> Cc: '[hidden email]'
> Subject: Re: [Rd] V2.9.0 changes [Sec=Unclassified]
>
> Troy Robertson wrote:
> > Hi all,
> >
> >
> >
> > Prefix: I am a frustrated Java coder in R.
>
> ah good, if you're frustrated with Java you'll find R very different ;)
>
> >
> >
> >
> > I am coding a medium sized ecosystem modelling program in R.  I have
> changed to using S4 objects and it has cost me an order of magnitude in
> execution speed over the functional model.  I cannot afford this penalty
> and have found that it is the result of all the passing-by-value of
> objects.
> >
> >
> >
> > I see that you can now safely inherit from environment in V2.9.0.
> >
> > That got me all excited that I would now be able to pass objects by
> reference.
> >
> >
> >
> > But...
> >
> > That doesn't seem to be the case.
> >
> > It only seem that passing an environment which holds the object allows
> for pass-by-reference and that passing an object which inherits from
> environment doesn't.
> >
> > Why is this the case, either an object inherits the properties of its
> parent or it doesn't.
>
> The object inherits slots from it's parent, and the methods defined on
> the parent class. Maybe this example helps?
>
> setClass("Ticker", contains=".environment")
>
> ## initialize essential, so each instance gets its own environment
> setMethod(initialize, "Ticker",
>           function(.Object, ..., .xData=new.env(parent=emptyenv()))
> {
>     .xData[["count"]] <- 0
>     callNextMethod(.Object, ..., .xData=.xData)
> })
>
> ## tick: increment (private) counter by n
> setGeneric("tick", function(reference, n=1L) standardGeneric("tick"),
>            signature="reference")
>
> setMethod(tick, "Ticker", function(reference, n=1L) {
>     reference[["count"]] <- reference[["count"]] + n
> })
>
> ## tock: report current value of counter
> setGeneric("tock", function(reference) standardGeneric("tock"))
>
> setMethod(tock, "Ticker", function(reference) {
>     reference[["count"]]
> })
>
> and then
>
> > e <- new("Ticker")
> > tock(e)
> [1] 0
> > tick(e); tick(e, 10); tock(e)
> [1] 11
> > f <- e
> > tock(f); tick(e); tock(f)
> [1] 11
> [1] 12
>
> The data inside .environment could be structured, too, using S4.
> Probably it would be more appropriate to have the environment as a slot,
> rather the class that is being extended. And in terms of inherited
> 'properties', e.g., the "[[" function as defined on environments is
> available
>
> > e[["count"]]
>
> Of course I haven't seen your code, but a different interpretation of
> your performance issues is that, within the rules of S4, you've chosen
> to implement functionality in an inefficient way. So it might be
> instructive to step back a bit and try to reformulate your data
> structures and methods. This is hard to do.
>
> Martin
>
> >
> > Has anyone else had a play with this?  Or have I got it all wrong.
> >
> >
> >
> > I tried the below:
> >
> > ------------------------------------------------------------------------
> -----------------
> >
> > setClass('foo', representation=representation(stuff='list',
> bar='numeric'),
> >
> >                      prototype=list(stuff=list(), bar=0),
> >
> >                      contains='.environment')
> >
> >
> >
> > setGeneric('doit', function(.Object, newfoo='environment')
> standardGeneric('doit'))
> >
> >
> >
> > setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
> >
> >
> >
> > z <- new('foo')
> >
> >
> >
> > z@stuff$x <- new('foo')
> >
> >
> >
> > doit(z,z@stuff$x)
> >
> >
> >
> > z@stuff$x@bar
> >
> >
> >
> > [1] 0
> >
> > ------------------------------------------------------------------------
> ------------------
> >
> >
> >
> > Can anyone help with a better way of doing this.
> >
> > I'm trying to avoid all the indirection of packing and unpacking
> environments for passing.
> >
> >
> >
> >
> >
> > Thanks heaps
> >
> >
> >
> > Troy
> >
> >
> >
> >
> >
> > Troy Robertson
> >
> > Database and Computing Support Provider
> >
> > Southern Ocean Ecosystems, ERM/Fish
> >
> > Australian Antarctic Division
> >
> > Channel Highway, Kingston 7050
> >
> > PH: 03 62323571
> >
> > [hidden email]
> >
> >
> >
> >
> >
> __________________________________________________________________________
> _
> >
> >     Australian Antarctic Division - Commonwealth of Australia
> > IMPORTANT: This transmission is intended for the addressee only. If you
> are not the
> > intended recipient, you are notified that use or dissemination of this
> communication is
> > strictly prohibited by Commonwealth law. If you have received this
> transmission in error,
> > please notify the sender immediately by e-mail or by telephoning +61 3
> 6232 3209 and
> > DELETE the message.
> >         Visit our web site at http://www.antarctica.gov.au/
> >
> __________________________________________________________________________
> _
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel

___________________________________________________________________________

    Australian Antarctic Division - Commonwealth of Australia
IMPORTANT: This transmission is intended for the addressee only. If you are not the
intended recipient, you are notified that use or dissemination of this communication is
strictly prohibited by Commonwealth law. If you have received this transmission in error,
please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
DELETE the message.
        Visit our web site at http://www.antarctica.gov.au/
___________________________________________________________________________

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Martin Maechler
In reply to this post by Troy Robertson
>>>>> "TR" == Troy Robertson <[hidden email]>
>>>>>     on Wed, 24 Jun 2009 16:35:29 +1000 writes:

    TR> Yes, I had looked at R.oo, S4 and proto before beginning coding. I had initially assumed that S4 was an enhancement of or replacement to R.oo that was implemented at a lower level and had decided to go with the 'future' of OO in R.
    TR> These assumptions were not necessarily correct.

>From the view of the R core team,  
S4  *is* ``the future of OO in R''

But then, as professional statisticians, we should consider the
famous
   >>> Prediction is very difficult, especially about the future <<
attributed to Physics Nobel Prize winner Niels Bohr.

---

Martin Maechler, ETH Zurich & R-core


    TR> Troy

    TR> Troy Robertson
    TR> Database and Computing Support Provider
    TR> Southern Ocean Ecosystems, ERM/Fish
    TR> Australian Antarctic Division
    TR> Channel Highway, Kingston 7050
    TR> PH: 03 62323571
    TR> [hidden email]


    >> -----Original Message-----
    >> From: Antonio, Fabio Di Narzo [mailto:[hidden email]]
    >> Sent: Tuesday, 23 June 2009 6:22 PM
    >> To: Troy Robertson
    >> Cc: [hidden email]
    >> Subject: Re: [Rd] V2.9.0 changes [Sec=Unclassified]
    >>
    >> Not a direct answer to your question, but...
    >> You might consider using the R.oo package, from H. Bengtsson. It's
    >> very stable, written in pure R, and cleanly allows you to do
    >> pass-by-reference OO programming, with no tricks.
    >>
    >> HTH,
    >> af
    >>
    >> 2009/6/23 Troy Robertson <[hidden email]>:
    >> > Hi all,
    >> >
    >> >
    >> >
    >> > Prefix: I am a frustrated Java coder in R.
    >> >
    >> >
    >> >
    >> > I am coding a medium sized ecosystem modelling program in R.  I have
    >> changed to using S4 objects and it has cost me an order of magnitude in
    >> execution speed over the functional model.  I cannot afford this penalty
    >> and have found that it is the result of all the passing-by-value of
    >> objects.
    >> >
    >> >
    >> >
    >> > I see that you can now safely inherit from environment in V2.9.0.
    >> >
    >> > That got me all excited that I would now be able to pass objects by
    >> reference.
    >> >
    >> >
    >> >
    >> > But...
    >> >
    >> > That doesn't seem to be the case.
    >> >
    >> > It only seem that passing an environment which holds the object allows
    >> for pass-by-reference and that passing an object which inherits from
    >> environment doesn't.
    >> >
    >> > Why is this the case, either an object inherits the properties of its
    >> parent or it doesn't.
    >> >
    >> > Has anyone else had a play with this?  Or have I got it all wrong.
    >> >
    >> >
    >> >
    >> > I tried the below:
    >> >
    >> > ------------------------------------------------------------------------
    >> -----------------
    >> >
    >> > setClass('foo', representation=representation(stuff='list',
    >> bar='numeric'),
    >> >
    >> >                     prototype=list(stuff=list(), bar=0),
    >> >
    >> >                     contains='.environment')
    >> >
    >> >
    >> >
    >> > setGeneric('doit', function(.Object, newfoo='environment')
    >> standardGeneric('doit'))
    >> >
    >> >
    >> >
    >> > setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
    >> >
    >> >
    >> >
    >> > z <- new('foo')
    >> >
    >> >
    >> >
    >> > z@stuff$x <- new('foo')
    >> >
    >> >
    >> >
    >> > doit(z,z@stuff$x)
    >> >
    >> >
    >> >
    >> > z@stuff$x@bar
    >> >
    >> >
    >> >
    >> > [1] 0
    >> >
    >> > ------------------------------------------------------------------------
    >> ------------------
    >> >
    >> >
    >> >
    >> > Can anyone help with a better way of doing this.
    >> >
    >> > I'm trying to avoid all the indirection of packing and unpacking
    >> environments for passing.
    >> >
    >> >
    >> >
    >> >
    >> >
    >> > Thanks heaps
    >> >
    >> > Troy
    >> >

    ..............


    >> --
    >> Antonio, Fabio Di Narzo
    >> Ph.D. student at
    >> Department of Statistical Sciences
    >> University of Bologna, Italy
    TR> ___________________________________________________________________________

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Peter Dalgaard
Martin Maechler wrote:
>
> But then, as professional statisticians, we should consider the
> famous
>    >>> Prediction is very difficult, especially about the future <<
> attributed to Physics Nobel Prize winner Niels Bohr.

...and quotations are even more difficult!

A number of people are known NOT to be the source of that one, including
Bohr, Robert Storm Petersen, and former minister of culture Steincke (in
his memoirs, he mentions it being used in parliament some time during
1935--1939, but he forgot by whom). One source has "Markus M. Ronner,
and others" (he was born in 1938, though!)

--
   O__  ---- Peter Dalgaard             Øster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark      Ph:  (+45) 35327918
~~~~~~~~~~ - ([hidden email])              FAX: (+45) 35327907

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Quotations (was Re: V2.9.0 changes [Sec=Unclassified])

Kevin R. Coombes
Also attributed (incorrectly) to both Yogi Berra and Mark Twain.  The
Twain attribution leads (because he published numerous essays and short
stories in The Galaxy magazine) to one possible (and documentable) source:

   "[I]f we found a character in a novel represented as habitually
    uttering true predictions of the future, we should cry out at once
    against the improbability."
       – Charles Astor Bristed (aka Carl Benson), Casual Cogitations,
          The Galaxy, 1873; 16:196–201.



Peter Dalgaard wrote:

> Martin Maechler wrote:
>  
>> But then, as professional statisticians, we should consider the
>> famous
>>    >>> Prediction is very difficult, especially about the future <<
>> attributed to Physics Nobel Prize winner Niels Bohr.
>>    
>
> ...and quotations are even more difficult!
>
> A number of people are known NOT to be the source of that one, including
> Bohr, Robert Storm Petersen, and former minister of culture Steincke (in
> his memoirs, he mentions it being used in parliament some time during
> 1935--1939, but he forgot by whom). One source has "Markus M. Ronner,
> and others" (he was born in 1938, though!)
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Henrik Bengtsson
In reply to this post by Martin Maechler
On Wed, Jun 24, 2009 at 4:27 AM, Martin
Maechler<[hidden email]> wrote:
>>>>>> "TR" == Troy Robertson <[hidden email]>
>>>>>>     on Wed, 24 Jun 2009 16:35:29 +1000 writes:
>
>    TR> Yes, I had looked at R.oo, S4 and proto before beginning coding. I had initially assumed that S4 was an enhancement of or replacement to R.oo that was implemented at a lower level and had decided to go with the 'future' of OO in R.
>    TR> These assumptions were not necessarily correct.
>
> >From the view of the R core team,
> S4  *is* ``the future of OO in R''

...and until we're there, R.oo (S3) will do it for you.

/Henrik

>
> But then, as professional statisticians, we should consider the
> famous
>   >>> Prediction is very difficult, especially about the future <<
> attributed to Physics Nobel Prize winner Niels Bohr.
>
> ---
>
> Martin Maechler, ETH Zurich & R-core
>
>
>    TR> Troy
>
>    TR> Troy Robertson
>    TR> Database and Computing Support Provider
>    TR> Southern Ocean Ecosystems, ERM/Fish
>    TR> Australian Antarctic Division
>    TR> Channel Highway, Kingston 7050
>    TR> PH: 03 62323571
>    TR> [hidden email]
>
>
>    >> -----Original Message-----
>    >> From: Antonio, Fabio Di Narzo [mailto:[hidden email]]
>    >> Sent: Tuesday, 23 June 2009 6:22 PM
>    >> To: Troy Robertson
>    >> Cc: [hidden email]
>    >> Subject: Re: [Rd] V2.9.0 changes [Sec=Unclassified]
>    >>
>    >> Not a direct answer to your question, but...
>    >> You might consider using the R.oo package, from H. Bengtsson. It's
>    >> very stable, written in pure R, and cleanly allows you to do
>    >> pass-by-reference OO programming, with no tricks.
>    >>
>    >> HTH,
>    >> af
>    >>
>    >> 2009/6/23 Troy Robertson <[hidden email]>:
>    >> > Hi all,
>    >> >
>    >> >
>    >> >
>    >> > Prefix: I am a frustrated Java coder in R.
>    >> >
>    >> >
>    >> >
>    >> > I am coding a medium sized ecosystem modelling program in R.  I have
>    >> changed to using S4 objects and it has cost me an order of magnitude in
>    >> execution speed over the functional model.  I cannot afford this penalty
>    >> and have found that it is the result of all the passing-by-value of
>    >> objects.
>    >> >
>    >> >
>    >> >
>    >> > I see that you can now safely inherit from environment in V2.9.0.
>    >> >
>    >> > That got me all excited that I would now be able to pass objects by
>    >> reference.
>    >> >
>    >> >
>    >> >
>    >> > But...
>    >> >
>    >> > That doesn't seem to be the case.
>    >> >
>    >> > It only seem that passing an environment which holds the object allows
>    >> for pass-by-reference and that passing an object which inherits from
>    >> environment doesn't.
>    >> >
>    >> > Why is this the case, either an object inherits the properties of its
>    >> parent or it doesn't.
>    >> >
>    >> > Has anyone else had a play with this?  Or have I got it all wrong.
>    >> >
>    >> >
>    >> >
>    >> > I tried the below:
>    >> >
>    >> > ------------------------------------------------------------------------
>    >> -----------------
>    >> >
>    >> > setClass('foo', representation=representation(stuff='list',
>    >> bar='numeric'),
>    >> >
>    >> >                     prototype=list(stuff=list(), bar=0),
>    >> >
>    >> >                     contains='.environment')
>    >> >
>    >> >
>    >> >
>    >> > setGeneric('doit', function(.Object, newfoo='environment')
>    >> standardGeneric('doit'))
>    >> >
>    >> >
>    >> >
>    >> > setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
>    >> >
>    >> >
>    >> >
>    >> > z <- new('foo')
>    >> >
>    >> >
>    >> >
>    >> > z@stuff$x <- new('foo')
>    >> >
>    >> >
>    >> >
>    >> > doit(z,z@stuff$x)
>    >> >
>    >> >
>    >> >
>    >> > z@stuff$x@bar
>    >> >
>    >> >
>    >> >
>    >> > [1] 0
>    >> >
>    >> > ------------------------------------------------------------------------
>    >> ------------------
>    >> >
>    >> >
>    >> >
>    >> > Can anyone help with a better way of doing this.
>    >> >
>    >> > I'm trying to avoid all the indirection of packing and unpacking
>    >> environments for passing.
>    >> >
>    >> >
>    >> >
>    >> >
>    >> >
>    >> > Thanks heaps
>    >> >
>    >> > Troy
>    >> >
>
>    ..............
>
>
>    >> --
>    >> Antonio, Fabio Di Narzo
>    >> Ph.D. student at
>    >> Department of Statistical Sciences
>    >> University of Bologna, Italy
>    TR> ___________________________________________________________________________
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Troy Robertson
In reply to this post by Martin Morgan
Well...

My performance problems were in the pass-by-value semantics of R.

I have just changed my classes to inherit from .environment and then moved data members from S4 slots to the .xData objects as Martin suggested.

That meant I could remove all my returns and assignments on all method calls.

This has sped execution time for my model up by more than an order of magnitude. Eg one test simulation from 1931 secs down to 175 secs.

Not bad seeing as though the class structure, functionality and logic has not been touched.

I really do think S4 could benfit from having its slots stored in environment when the class enherits from .environment.  It would be a lot more sensible if my data members were still declared as S4 slots instead of having to hide them in .xData

Troy


Troy Robertson
Database and Computing Support Provider
Southern Ocean Ecosystems, ERM/Fish
Australian Antarctic Division
Channel Highway, Kingston 7050
PH: 03 62323571
[hidden email]


> -----Original Message-----
> From: Martin Morgan [mailto:[hidden email]]
> Sent: Tuesday, 23 June 2009 11:25 PM
> To: Troy Robertson
> Cc: '[hidden email]'
> Subject: Re: [Rd] V2.9.0 changes [Sec=Unclassified]
>
> Troy Robertson wrote:
> > Hi all,
> >
> >
> >
> > Prefix: I am a frustrated Java coder in R.
>
> ah good, if you're frustrated with Java you'll find R very different ;)
>
> >
> >
> >
> > I am coding a medium sized ecosystem modelling program in R.  I have
> changed to using S4 objects and it has cost me an order of magnitude in
> execution speed over the functional model.  I cannot afford this penalty
> and have found that it is the result of all the passing-by-value of
> objects.
> >
> >
> >
> > I see that you can now safely inherit from environment in V2.9.0.
> >
> > That got me all excited that I would now be able to pass objects by
> reference.
> >
> >
> >
> > But...
> >
> > That doesn't seem to be the case.
> >
> > It only seem that passing an environment which holds the object allows
> for pass-by-reference and that passing an object which inherits from
> environment doesn't.
> >
> > Why is this the case, either an object inherits the properties of its
> parent or it doesn't.
>
> The object inherits slots from it's parent, and the methods defined on
> the parent class. Maybe this example helps?
>
> setClass("Ticker", contains=".environment")
>
> ## initialize essential, so each instance gets its own environment
> setMethod(initialize, "Ticker",
>           function(.Object, ..., .xData=new.env(parent=emptyenv()))
> {
>     .xData[["count"]] <- 0
>     callNextMethod(.Object, ..., .xData=.xData)
> })
>
> ## tick: increment (private) counter by n
> setGeneric("tick", function(reference, n=1L) standardGeneric("tick"),
>            signature="reference")
>
> setMethod(tick, "Ticker", function(reference, n=1L) {
>     reference[["count"]] <- reference[["count"]] + n
> })
>
> ## tock: report current value of counter
> setGeneric("tock", function(reference) standardGeneric("tock"))
>
> setMethod(tock, "Ticker", function(reference) {
>     reference[["count"]]
> })
>
> and then
>
> > e <- new("Ticker")
> > tock(e)
> [1] 0
> > tick(e); tick(e, 10); tock(e)
> [1] 11
> > f <- e
> > tock(f); tick(e); tock(f)
> [1] 11
> [1] 12
>
> The data inside .environment could be structured, too, using S4.
> Probably it would be more appropriate to have the environment as a slot,
> rather the class that is being extended. And in terms of inherited
> 'properties', e.g., the "[[" function as defined on environments is
> available
>
> > e[["count"]]
>
> Of course I haven't seen your code, but a different interpretation of
> your performance issues is that, within the rules of S4, you've chosen
> to implement functionality in an inefficient way. So it might be
> instructive to step back a bit and try to reformulate your data
> structures and methods. This is hard to do.
>
> Martin
>
> >
> > Has anyone else had a play with this?  Or have I got it all wrong.
> >
> >
> >
> > I tried the below:
> >
> > ------------------------------------------------------------------------
> -----------------
> >
> > setClass('foo', representation=representation(stuff='list',
> bar='numeric'),
> >
> >                      prototype=list(stuff=list(), bar=0),
> >
> >                      contains='.environment')
> >
> >
> >
> > setGeneric('doit', function(.Object, newfoo='environment')
> standardGeneric('doit'))
> >
> >
> >
> > setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
> >
> >
> >
> > z <- new('foo')
> >
> >
> >
> > z@stuff$x <- new('foo')
> >
> >
> >
> > doit(z,z@stuff$x)
> >
> >
> >
> > z@stuff$x@bar
> >
> >
> >
> > [1] 0
> >
> > ------------------------------------------------------------------------
> ------------------
> >
> >
> >
> > Can anyone help with a better way of doing this.
> >
> > I'm trying to avoid all the indirection of packing and unpacking
> environments for passing.
> >
> >
> >
> >
> >
> > Thanks heaps
> >
> >
> >
> > Troy
> >
> >
> >
> >
> >
> > Troy Robertson
> >
> > Database and Computing Support Provider
> >
> > Southern Ocean Ecosystems, ERM/Fish
> >
> > Australian Antarctic Division
> >
> > Channel Highway, Kingston 7050
> >
> > PH: 03 62323571
> >
> > [hidden email]
> >
> >
> >
> >
> >
> __________________________________________________________________________
> _
> >
> >     Australian Antarctic Division - Commonwealth of Australia
> > IMPORTANT: This transmission is intended for the addressee only. If you
> are not the
> > intended recipient, you are notified that use or dissemination of this
> communication is
> > strictly prohibited by Commonwealth law. If you have received this
> transmission in error,
> > please notify the sender immediately by e-mail or by telephoning +61 3
> 6232 3209 and
> > DELETE the message.
> >         Visit our web site at http://www.antarctica.gov.au/
> >
> __________________________________________________________________________
> _
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel

___________________________________________________________________________

    Australian Antarctic Division - Commonwealth of Australia
IMPORTANT: This transmission is intended for the addressee only. If you are not the
intended recipient, you are notified that use or dissemination of this communication is
strictly prohibited by Commonwealth law. If you have received this transmission in error,
please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
DELETE the message.
        Visit our web site at http://www.antarctica.gov.au/
___________________________________________________________________________

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Martin Morgan
Troy Robertson wrote:
> Well...
>
> My performance problems were in the pass-by-value semantics of R.
>
> I have just changed my classes to inherit from .environment and then moved data members from S4 slots to the .xData objects as Martin suggested.

Actually, I had hoped the take-home message would be in the final paragraph:

>> Of course I haven't seen your code, but a different interpretation of
>> your performance issues is that, within the rules of S4, you've chosen
>> to implement functionality in an inefficient way. So it might be
>> instructive to step back a bit and try to reformulate your data
>> structures and methods. This is hard to do.

Martin

> That meant I could remove all my returns and assignments on all method calls.
>
> This has sped execution time for my model up by more than an order of magnitude. Eg one test simulation from 1931 secs down to 175 secs.
>
> Not bad seeing as though the class structure, functionality and logic has not been touched.
>
> I really do think S4 could benfit from having its slots stored in environment when the class enherits from .environment.  It would be a lot more sensible if my data members were still declared as S4 slots instead of having to hide them in .xData
>
> Troy
>
>
> Troy Robertson
> Database and Computing Support Provider
> Southern Ocean Ecosystems, ERM/Fish
> Australian Antarctic Division
> Channel Highway, Kingston 7050
> PH: 03 62323571
> [hidden email]
>
>
>> -----Original Message-----
>> From: Martin Morgan [mailto:[hidden email]]
>> Sent: Tuesday, 23 June 2009 11:25 PM
>> To: Troy Robertson
>> Cc: '[hidden email]'
>> Subject: Re: [Rd] V2.9.0 changes [Sec=Unclassified]
>>
>> Troy Robertson wrote:
>>> Hi all,
>>>
>>>
>>>
>>> Prefix: I am a frustrated Java coder in R.
>> ah good, if you're frustrated with Java you'll find R very different ;)
>>
>>>
>>>
>>> I am coding a medium sized ecosystem modelling program in R.  I have
>> changed to using S4 objects and it has cost me an order of magnitude in
>> execution speed over the functional model.  I cannot afford this penalty
>> and have found that it is the result of all the passing-by-value of
>> objects.
>>>
>>>
>>> I see that you can now safely inherit from environment in V2.9.0.
>>>
>>> That got me all excited that I would now be able to pass objects by
>> reference.
>>>
>>>
>>> But...
>>>
>>> That doesn't seem to be the case.
>>>
>>> It only seem that passing an environment which holds the object allows
>> for pass-by-reference and that passing an object which inherits from
>> environment doesn't.
>>> Why is this the case, either an object inherits the properties of its
>> parent or it doesn't.
>>
>> The object inherits slots from it's parent, and the methods defined on
>> the parent class. Maybe this example helps?
>>
>> setClass("Ticker", contains=".environment")
>>
>> ## initialize essential, so each instance gets its own environment
>> setMethod(initialize, "Ticker",
>>           function(.Object, ..., .xData=new.env(parent=emptyenv()))
>> {
>>     .xData[["count"]] <- 0
>>     callNextMethod(.Object, ..., .xData=.xData)
>> })
>>
>> ## tick: increment (private) counter by n
>> setGeneric("tick", function(reference, n=1L) standardGeneric("tick"),
>>            signature="reference")
>>
>> setMethod(tick, "Ticker", function(reference, n=1L) {
>>     reference[["count"]] <- reference[["count"]] + n
>> })
>>
>> ## tock: report current value of counter
>> setGeneric("tock", function(reference) standardGeneric("tock"))
>>
>> setMethod(tock, "Ticker", function(reference) {
>>     reference[["count"]]
>> })
>>
>> and then
>>
>>> e <- new("Ticker")
>>> tock(e)
>> [1] 0
>>> tick(e); tick(e, 10); tock(e)
>> [1] 11
>>> f <- e
>>> tock(f); tick(e); tock(f)
>> [1] 11
>> [1] 12
>>
>> The data inside .environment could be structured, too, using S4.
>> Probably it would be more appropriate to have the environment as a slot,
>> rather the class that is being extended. And in terms of inherited
>> 'properties', e.g., the "[[" function as defined on environments is
>> available
>>
>>> e[["count"]]
>> Of course I haven't seen your code, but a different interpretation of
>> your performance issues is that, within the rules of S4, you've chosen
>> to implement functionality in an inefficient way. So it might be
>> instructive to step back a bit and try to reformulate your data
>> structures and methods. This is hard to do.
>>
>> Martin
>>
>>> Has anyone else had a play with this?  Or have I got it all wrong.
>>>
>>>
>>>
>>> I tried the below:
>>>
>>> ------------------------------------------------------------------------
>> -----------------
>>> setClass('foo', representation=representation(stuff='list',
>> bar='numeric'),
>>>                      prototype=list(stuff=list(), bar=0),
>>>
>>>                      contains='.environment')
>>>
>>>
>>>
>>> setGeneric('doit', function(.Object, newfoo='environment')
>> standardGeneric('doit'))
>>>
>>>
>>> setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
>>>
>>>
>>>
>>> z <- new('foo')
>>>
>>>
>>>
>>> z@stuff$x <- new('foo')
>>>
>>>
>>>
>>> doit(z,z@stuff$x)
>>>
>>>
>>>
>>> z@stuff$x@bar
>>>
>>>
>>>
>>> [1] 0
>>>
>>> ------------------------------------------------------------------------
>> ------------------
>>>
>>>
>>> Can anyone help with a better way of doing this.
>>>
>>> I'm trying to avoid all the indirection of packing and unpacking
>> environments for passing.
>>>
>>>
>>>
>>>
>>> Thanks heaps
>>>
>>>
>>>
>>> Troy
>>>
>>>
>>>
>>>
>>>
>>> Troy Robertson
>>>
>>> Database and Computing Support Provider
>>>
>>> Southern Ocean Ecosystems, ERM/Fish
>>>
>>> Australian Antarctic Division
>>>
>>> Channel Highway, Kingston 7050
>>>
>>> PH: 03 62323571
>>>
>>> [hidden email]
>>>
>>>
>>>
>>>
>>>
>> __________________________________________________________________________
>> _
>>>     Australian Antarctic Division - Commonwealth of Australia
>>> IMPORTANT: This transmission is intended for the addressee only. If you
>> are not the
>>> intended recipient, you are notified that use or dissemination of this
>> communication is
>>> strictly prohibited by Commonwealth law. If you have received this
>> transmission in error,
>>> please notify the sender immediately by e-mail or by telephoning +61 3
>> 6232 3209 and
>>> DELETE the message.
>>>         Visit our web site at http://www.antarctica.gov.au/
>>>
>> __________________________________________________________________________
>> _
>>>       [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> ___________________________________________________________________________
>
>     Australian Antarctic Division - Commonwealth of Australia
> IMPORTANT: This transmission is intended for the addressee only. If you are not the
> intended recipient, you are notified that use or dissemination of this communication is
> strictly prohibited by Commonwealth law. If you have received this transmission in error,
> please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
> DELETE the message.
>         Visit our web site at http://www.antarctica.gov.au/
> ___________________________________________________________________________

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Gabor Grothendieck
In reply to this post by Troy Robertson
On Thu, Jul 2, 2009 at 1:37 AM, Troy Robertson<[hidden email]> wrote:
> Well...
>
> My performance problems were in the pass-by-value semantics of R.
>
> I have just changed my classes to inherit from .environment and then moved data members from S4 slots to the .xData objects as Martin suggested.
>

Note that the R.oo and proto packages already use environments for
their storage. e.g.

library(proto)
p <- proto(a = 1, incr = function(.) .$a <- .$a + 1)
class(p) # c("proto", "environment")

p$a # 1
p$incr()
p$a # 2

p$ls() #  c("a", "incr")
ls(p) # same

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

OOP performance, was: V2.9.0 changes

Thomas Petzoldt
In reply to this post by Troy Robertson
Hi Troy,

first of all a question, what kind of ecosystem models are you
developing in R? Differential equations or individual-based?

Your write that you are a frustrated Java developer in R. I have a
similar experience, however I still like JAVA, and I'm now more happy
with R as it is much more efficient (i.e. sum(programming + runtime))
for the things I usually do: ecological data analysis and modelling.

After using functional R quite a time and Java in parallel
I had the same idea, to make R more JAVA like and to model ecosystems in
an object oriented manner. At that time I took a look into R.oo (thanks
Henrik Bengtssson) and was one of the Co-authors of proto. I still think
that R.oo is very good and that proto is a cool idea, but finally I
switched to the recommended S4 for my ecological simulation package.

Note also, that my solution was *not* to model the ecosystems as objects
(habitat - populations- individuals), but instead to model ecological
models (equations, inputs, parameters, time steps, outputs, ...).

This works quite well with S4. A speed test (see useR!2006 poster on
http://simecol.r-forge.r-project.org/) showed that all OOP flavours had
quite comparable performance.

The only thing I have to have in mind are a few rules:

- avoid unnecessary copying of large objects. Sometimes it helps to
prefer matrices over data frames.

- use vectorization. This means for an individual-based model that one
has to re-think how to model an individual: not "many [S4] objects"
like in JAVA, but R structures (arrays, lists, data frames) where
vectorized functions (e.g. arithmetics or subset) can work with.

- avoid interpolation (i.e. approx) and if unavoidable, minimize the tables.

If all these things do not help, I write core functions in C (others use
Fortran). This can be done in a mixed style and even a full C to C
communication is possible (see the deSolve documentation how to do this
with differential equation models).


Thomas P.



--
Thomas Petzoldt
Technische Universitaet Dresden
Institut fuer Hydrobiologie        [hidden email]
01062 Dresden                      http://tu-dresden.de/hydrobiologie/
GERMANY

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: V2.9.0 changes [Sec=Unclassified]

Troy Robertson
In reply to this post by Martin Morgan

> -----Original Message-----
> From: Martin Morgan [mailto:[hidden email]]
> Sent: Thursday, 2 July 2009 10:58 PM
> To: Troy Robertson
> Cc: '[hidden email]'
> Subject: Re: [Rd] V2.9.0 changes [Sec=Unclassified]
>
> Troy Robertson wrote:
> > Well...
> >
> > My performance problems were in the pass-by-value semantics of R.
> >
> > I have just changed my classes to inherit from .environment and then
> moved data members from S4 slots to the .xData objects as Martin
> suggested.
>
> Actually, I had hoped the take-home message would be in the final
> paragraph:
>
> >> Of course I haven't seen your code, but a different interpretation of
> >> your performance issues is that, within the rules of S4, you've chosen
> >> to implement functionality in an inefficient way. So it might be
> >> instructive to step back a bit and try to reformulate your data
> >> structures and methods. This is hard to do.
>

Yes, it just takes a little time and playing around to work out the rules of R (and S4) though, and how to use them to your advantage rather than be limited by them.  I know I am still probably not using this functional language in the best way, but there you go.


>
> > That meant I could remove all my returns and assignments on all method
> calls.
> >
> > This has sped execution time for my model up by more than an order of
> magnitude. Eg one test simulation from 1931 secs down to 175 secs.
> >
> > Not bad seeing as though the class structure, functionality and logic
> has not been touched.
> >
> > I really do think S4 could benfit from having its slots stored in
> environment when the class enherits from .environment.  It would be a lot
> more sensible if my data members were still declared as S4 slots instead
> of having to hide them in .xData
> >
> > Troy
> >
> >
> > Troy Robertson
> > Database and Computing Support Provider
> > Southern Ocean Ecosystems, ERM/Fish
> > Australian Antarctic Division
> > Channel Highway, Kingston 7050
> > PH: 03 62323571
> > [hidden email]
> >
> >
> >> -----Original Message-----
> >> From: Martin Morgan [mailto:[hidden email]]
> >> Sent: Tuesday, 23 June 2009 11:25 PM
> >> To: Troy Robertson
> >> Cc: '[hidden email]'
> >> Subject: Re: [Rd] V2.9.0 changes [Sec=Unclassified]
> >>
> >> Troy Robertson wrote:
> >>> Hi all,
> >>>
> >>>
> >>>
> >>> Prefix: I am a frustrated Java coder in R.
> >> ah good, if you're frustrated with Java you'll find R very different ;)
> >>
> >>>
> >>>
> >>> I am coding a medium sized ecosystem modelling program in R.  I have
> >> changed to using S4 objects and it has cost me an order of magnitude in
> >> execution speed over the functional model.  I cannot afford this
> penalty
> >> and have found that it is the result of all the passing-by-value of
> >> objects.
> >>>
> >>>
> >>> I see that you can now safely inherit from environment in V2.9.0.
> >>>
> >>> That got me all excited that I would now be able to pass objects by
> >> reference.
> >>>
> >>>
> >>> But...
> >>>
> >>> That doesn't seem to be the case.
> >>>
> >>> It only seem that passing an environment which holds the object allows
> >> for pass-by-reference and that passing an object which inherits from
> >> environment doesn't.
> >>> Why is this the case, either an object inherits the properties of its
> >> parent or it doesn't.
> >>
> >> The object inherits slots from it's parent, and the methods defined on
> >> the parent class. Maybe this example helps?
> >>
> >> setClass("Ticker", contains=".environment")
> >>
> >> ## initialize essential, so each instance gets its own environment
> >> setMethod(initialize, "Ticker",
> >>           function(.Object, ..., .xData=new.env(parent=emptyenv()))
> >> {
> >>     .xData[["count"]] <- 0
> >>     callNextMethod(.Object, ..., .xData=.xData)
> >> })
> >>
> >> ## tick: increment (private) counter by n
> >> setGeneric("tick", function(reference, n=1L) standardGeneric("tick"),
> >>            signature="reference")
> >>
> >> setMethod(tick, "Ticker", function(reference, n=1L) {
> >>     reference[["count"]] <- reference[["count"]] + n
> >> })
> >>
> >> ## tock: report current value of counter
> >> setGeneric("tock", function(reference) standardGeneric("tock"))
> >>
> >> setMethod(tock, "Ticker", function(reference) {
> >>     reference[["count"]]
> >> })
> >>
> >> and then
> >>
> >>> e <- new("Ticker")
> >>> tock(e)
> >> [1] 0
> >>> tick(e); tick(e, 10); tock(e)
> >> [1] 11
> >>> f <- e
> >>> tock(f); tick(e); tock(f)
> >> [1] 11
> >> [1] 12
> >>
> >> The data inside .environment could be structured, too, using S4.
> >> Probably it would be more appropriate to have the environment as a
> slot,
> >> rather the class that is being extended. And in terms of inherited
> >> 'properties', e.g., the "[[" function as defined on environments is
> >> available
> >>
> >>> e[["count"]]
> >> Of course I haven't seen your code, but a different interpretation of
> >> your performance issues is that, within the rules of S4, you've chosen
> >> to implement functionality in an inefficient way. So it might be
> >> instructive to step back a bit and try to reformulate your data
> >> structures and methods. This is hard to do.
> >>
> >> Martin
> >>
> >>> Has anyone else had a play with this?  Or have I got it all wrong.
> >>>
> >>>
> >>>
> >>> I tried the below:
> >>>
> >>> ----------------------------------------------------------------------
> --
> >> -----------------
> >>> setClass('foo', representation=representation(stuff='list',
> >> bar='numeric'),
> >>>                      prototype=list(stuff=list(), bar=0),
> >>>
> >>>                      contains='.environment')
> >>>
> >>>
> >>>
> >>> setGeneric('doit', function(.Object, newfoo='environment')
> >> standardGeneric('doit'))
> >>>
> >>>
> >>> setMethod('doit', 'foo', function(.Object, newfoo){newfoo@bar <- 10})
> >>>
> >>>
> >>>
> >>> z <- new('foo')
> >>>
> >>>
> >>>
> >>> z@stuff$x <- new('foo')
> >>>
> >>>
> >>>
> >>> doit(z,z@stuff$x)
> >>>
> >>>
> >>>
> >>> z@stuff$x@bar
> >>>
> >>>
> >>>
> >>> [1] 0
> >>>
> >>> ----------------------------------------------------------------------
> --
> >> ------------------
> >>>
> >>>
> >>> Can anyone help with a better way of doing this.
> >>>
> >>> I'm trying to avoid all the indirection of packing and unpacking
> >> environments for passing.
> >>>
> >>>
> >>>
> >>>
> >>> Thanks heaps
> >>>
> >>>
> >>>
> >>> Troy
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> Troy Robertson
> >>>
> >>> Database and Computing Support Provider
> >>>
> >>> Southern Ocean Ecosystems, ERM/Fish
> >>>
> >>> Australian Antarctic Division
> >>>
> >>> Channel Highway, Kingston 7050
> >>>
> >>> PH: 03 62323571
> >>>
> >>> [hidden email]
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> __________________________________________________________________________
> >> _
> >>>     Australian Antarctic Division - Commonwealth of Australia
> >>> IMPORTANT: This transmission is intended for the addressee only. If
> you
> >> are not the
> >>> intended recipient, you are notified that use or dissemination of this
> >> communication is
> >>> strictly prohibited by Commonwealth law. If you have received this
> >> transmission in error,
> >>> please notify the sender immediately by e-mail or by telephoning +61 3
> >> 6232 3209 and
> >>> DELETE the message.
> >>>         Visit our web site at http://www.antarctica.gov.au/
> >>>
> >>
> __________________________________________________________________________
> >> _
> >>>       [[alternative HTML version deleted]]
> >>>
> >>> ______________________________________________
> >>> [hidden email] mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> >
> __________________________________________________________________________
> _
> >
> >     Australian Antarctic Division - Commonwealth of Australia
> > IMPORTANT: This transmission is intended for the addressee only. If you
> are not the
> > intended recipient, you are notified that use or dissemination of this
> communication is
> > strictly prohibited by Commonwealth law. If you have received this
> transmission in error,
> > please notify the sender immediately by e-mail or by telephoning +61 3
> 6232 3209 and
> > DELETE the message.
> >         Visit our web site at http://www.antarctica.gov.au/
> >
> __________________________________________________________________________
> _

___________________________________________________________________________

    Australian Antarctic Division - Commonwealth of Australia
IMPORTANT: This transmission is intended for the addressee only. If you are not the
intended recipient, you are notified that use or dissemination of this communication is
strictly prohibited by Commonwealth law. If you have received this transmission in error,
please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
DELETE the message.
        Visit our web site at http://www.antarctica.gov.au/
___________________________________________________________________________

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: OOP performance, was: V2.9.0 changes [SEC=Unclassified]

Troy Robertson
In reply to this post by Thomas Petzoldt
Hi Thomas,

It is a population-based model, but I didn't develop the work.  I am just the programmer who has been given the job of coding it.  The goal is to allow for a plug and play type approach by users to construction of the model (of both elements and functionality).  Hence my focus on OO.

You are right about avoiding the copying of large objects.  That is what was killing things.  I am now working on vectorizing more of the number crunching and removing some of the nested for loops.  That should step things up a little too.

I do also need to investigate how to move some of the more expensive code to C.

Had a quick look at simecol which looks interesting.  Will point it out to my boss to check out too.

Thanks

Troy

> -----Original Message-----
> From: Thomas Petzoldt [mailto:[hidden email]]
> Sent: Friday, 3 July 2009 1:31 AM
> To: Troy Robertson
> Cc: '[hidden email]'
> Subject: OOP performance, was: [Rd] V2.9.0 changes [SEC=Unclassified]
>
> Hi Troy,
>
> first of all a question, what kind of ecosystem models are you
> developing in R? Differential equations or individual-based?
>
> Your write that you are a frustrated Java developer in R. I have a
> similar experience, however I still like JAVA, and I'm now more happy
> with R as it is much more efficient (i.e. sum(programming + runtime))
> for the things I usually do: ecological data analysis and modelling.
>
> After using functional R quite a time and Java in parallel
> I had the same idea, to make R more JAVA like and to model ecosystems in
> an object oriented manner. At that time I took a look into R.oo (thanks
> Henrik Bengtssson) and was one of the Co-authors of proto. I still think
> that R.oo is very good and that proto is a cool idea, but finally I
> switched to the recommended S4 for my ecological simulation package.
>
> Note also, that my solution was *not* to model the ecosystems as objects
> (habitat - populations- individuals), but instead to model ecological
> models (equations, inputs, parameters, time steps, outputs, ...).
>
> This works quite well with S4. A speed test (see useR!2006 poster on
> http://simecol.r-forge.r-project.org/) showed that all OOP flavours had
> quite comparable performance.
>
> The only thing I have to have in mind are a few rules:
>
> - avoid unnecessary copying of large objects. Sometimes it helps to
> prefer matrices over data frames.
>
> - use vectorization. This means for an individual-based model that one
> has to re-think how to model an individual: not "many [S4] objects"
> like in JAVA, but R structures (arrays, lists, data frames) where
> vectorized functions (e.g. arithmetics or subset) can work with.
>
> - avoid interpolation (i.e. approx) and if unavoidable, minimize the
> tables.
>
> If all these things do not help, I write core functions in C (others use
> Fortran). This can be done in a mixed style and even a full C to C
> communication is possible (see the deSolve documentation how to do this
> with differential equation models).
>
>
> Thomas P.
>
>
>
> --
> Thomas Petzoldt
> Technische Universitaet Dresden
> Institut fuer Hydrobiologie        [hidden email]
> 01062 Dresden                      http://tu-dresden.de/hydrobiologie/
> GERMANY
>
>
>
>

___________________________________________________________________________

    Australian Antarctic Division - Commonwealth of Australia
IMPORTANT: This transmission is intended for the addressee only. If you are not the
intended recipient, you are notified that use or dissemination of this communication is
strictly prohibited by Commonwealth law. If you have received this transmission in error,
please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
DELETE the message.
        Visit our web site at http://www.antarctica.gov.au/
___________________________________________________________________________

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: OOP performance, was: V2.9.0 changes

Gabor Grothendieck
In reply to this post by Thomas Petzoldt
In terms of performance if you want the fastest
performance in R go with S3 and if you want
even faster performance rewrite your inner loops
in C.  All the other approaches will usually be slower.
Also S3 is simple, elegant and will result in less code
and take you much less time to design, program and
debug.

For 100% R code, particularly for simulations,
proto can sometimes be even faster than pure R code based
S3 as proto supports hand optimizations that cannot readily
be done in other systems.  (For unoptimized code it would
be slower.)  The key trick is based on its ability
to separate dispatching from calling so that if method f and
object p are unchanged in a loop
   for(...) p$f(...)
then the loop can be rewritten
  f <- p$f; for(...) f(...)
Note that this still retains dynamic dispatch but
just factors it out of the loop.  With S3 the best you could
get would be for(...) f.p(...) where f is a method of class p
but this is really tantamount to not using OO at all since
no dispatch is done at all.

On Thu, Jul 2, 2009 at 11:31 AM, Thomas
Petzoldt<[hidden email]> wrote:

> Hi Troy,
>
> first of all a question, what kind of ecosystem models are you
> developing in R? Differential equations or individual-based?
>
> Your write that you are a frustrated Java developer in R. I have a
> similar experience, however I still like JAVA, and I'm now more happy
> with R as it is much more efficient (i.e. sum(programming + runtime))
> for the things I usually do: ecological data analysis and modelling.
>
> After using functional R quite a time and Java in parallel
> I had the same idea, to make R more JAVA like and to model ecosystems in
> an object oriented manner. At that time I took a look into R.oo (thanks
> Henrik Bengtssson) and was one of the Co-authors of proto. I still think
> that R.oo is very good and that proto is a cool idea, but finally I
> switched to the recommended S4 for my ecological simulation package.
>
> Note also, that my solution was *not* to model the ecosystems as objects
> (habitat - populations- individuals), but instead to model ecological
> models (equations, inputs, parameters, time steps, outputs, ...).
>
> This works quite well with S4. A speed test (see useR!2006 poster on
> http://simecol.r-forge.r-project.org/) showed that all OOP flavours had
> quite comparable performance.
>
> The only thing I have to have in mind are a few rules:
>
> - avoid unnecessary copying of large objects. Sometimes it helps to
> prefer matrices over data frames.
>
> - use vectorization. This means for an individual-based model that one
> has to re-think how to model an individual: not "many [S4] objects"
> like in JAVA, but R structures (arrays, lists, data frames) where
> vectorized functions (e.g. arithmetics or subset) can work with.
>
> - avoid interpolation (i.e. approx) and if unavoidable, minimize the tables.
>
> If all these things do not help, I write core functions in C (others use
> Fortran). This can be done in a mixed style and even a full C to C
> communication is possible (see the deSolve documentation how to do this
> with differential equation models).
>
>
> Thomas P.
>
>
>
> --
> Thomas Petzoldt
> Technische Universitaet Dresden
> Institut fuer Hydrobiologie        [hidden email]
> 01062 Dresden                      http://tu-dresden.de/hydrobiologie/
> GERMANY
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: OOP performance, was: V2.9.0 changes [SEC=Unclassified]

Troy Robertson
Hi Gabor,

Look, I agree with you about S3 and have at times wished I had chosen that path rather than S4.  It seems to do the things I struggle to find answers for with S4.  But..., knowing little about R before engaging with this project, I decided to go with the latest OO framework, S4.  I do now find that I am undoing some of it, such as the use of data member slots, in order to implement pass-by-ref via environments and improve performance.  But its all a learning experience.

Troy


> -----Original Message-----
> From: Gabor Grothendieck [mailto:[hidden email]]
> Sent: Friday, 3 July 2009 10:29 AM
> To: Troy Robertson; [hidden email]
> Subject: Re: [Rd] OOP performance, was: V2.9.0 changes [SEC=Unclassified]
>
> In terms of performance if you want the fastest
> performance in R go with S3 and if you want
> even faster performance rewrite your inner loops
> in C.  All the other approaches will usually be slower.
> Also S3 is simple, elegant and will result in less code
> and take you much less time to design, program and
> debug.
>
> For 100% R code, particularly for simulations,
> proto can sometimes be even faster than pure R code based
> S3 as proto supports hand optimizations that cannot readily
> be done in other systems.  (For unoptimized code it would
> be slower.)  The key trick is based on its ability
> to separate dispatching from calling so that if method f and
> object p are unchanged in a loop
>    for(...) p$f(...)
> then the loop can be rewritten
>   f <- p$f; for(...) f(...)
> Note that this still retains dynamic dispatch but
> just factors it out of the loop.  With S3 the best you could
> get would be for(...) f.p(...) where f is a method of class p
> but this is really tantamount to not using OO at all since
> no dispatch is done at all.
>
> On Thu, Jul 2, 2009 at 11:31 AM, Thomas
> Petzoldt<[hidden email]> wrote:
> > Hi Troy,
> >
> > first of all a question, what kind of ecosystem models are you
> > developing in R? Differential equations or individual-based?
> >
> > Your write that you are a frustrated Java developer in R. I have a
> > similar experience, however I still like JAVA, and I'm now more happy
> > with R as it is much more efficient (i.e. sum(programming + runtime))
> > for the things I usually do: ecological data analysis and modelling.
> >
> > After using functional R quite a time and Java in parallel
> > I had the same idea, to make R more JAVA like and to model ecosystems in
> > an object oriented manner. At that time I took a look into R.oo (thanks
> > Henrik Bengtssson) and was one of the Co-authors of proto. I still think
> > that R.oo is very good and that proto is a cool idea, but finally I
> > switched to the recommended S4 for my ecological simulation package.
> >
> > Note also, that my solution was *not* to model the ecosystems as objects
> > (habitat - populations- individuals), but instead to model ecological
> > models (equations, inputs, parameters, time steps, outputs, ...).
> >
> > This works quite well with S4. A speed test (see useR!2006 poster on
> > http://simecol.r-forge.r-project.org/) showed that all OOP flavours had
> > quite comparable performance.
> >
> > The only thing I have to have in mind are a few rules:
> >
> > - avoid unnecessary copying of large objects. Sometimes it helps to
> > prefer matrices over data frames.
> >
> > - use vectorization. This means for an individual-based model that one
> > has to re-think how to model an individual: not "many [S4] objects"
> > like in JAVA, but R structures (arrays, lists, data frames) where
> > vectorized functions (e.g. arithmetics or subset) can work with.
> >
> > - avoid interpolation (i.e. approx) and if unavoidable, minimize the
> tables.
> >
> > If all these things do not help, I write core functions in C (others use
> > Fortran). This can be done in a mixed style and even a full C to C
> > communication is possible (see the deSolve documentation how to do this
> > with differential equation models).
> >
> >
> > Thomas P.
> >
> >
> >
> > --
> > Thomas Petzoldt
> > Technische Universitaet Dresden
> > Institut fuer Hydrobiologie        [hidden email]
> > 01062 Dresden                      http://tu-dresden.de/hydrobiologie/
> > GERMANY
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
___________________________________________________________________________

    Australian Antarctic Division - Commonwealth of Australia
IMPORTANT: This transmission is intended for the addressee only. If you are not the
intended recipient, you are notified that use or dissemination of this communication is
strictly prohibited by Commonwealth law. If you have received this transmission in error,
please notify the sender immediately by e-mail or by telephoning +61 3 6232 3209 and
DELETE the message.
        Visit our web site at http://www.antarctica.gov.au/
___________________________________________________________________________

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel