profiling C code

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

profiling C code

Ross Boylan
Does anyone have any advice about profiling C/C++ code in a package
under R?  Does R need to be built specially for this to work?

The FAQ has some entries about profiling but they cover R level
profiling; I'm try to get at the C++ code I've written that is called
from R.

Primary target is Mac OS X.

Thanks.

Ross Boylan

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: profiling C code

Liaw, Andy
The last time I tried I didn't have much luck.  The gprof manual I could
find seems to indicate that it can not profile code that are dynamically
loaded.  (I was trying on Linux.)  The R source seems to hint otherwise.
I'd very much appreciate pointers as well.

Andy

From: Ross Boylan

>
> Does anyone have any advice about profiling C/C++ code in a package
> under R?  Does R need to be built specially for this to work?
>
> The FAQ has some entries about profiling but they cover R level
> profiling; I'm try to get at the C++ code I've written that is called
> from R.
>
> Primary target is Mac OS X.
>
> Thanks.
>
> Ross Boylan
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: profiling C code

Torsten Hothorn

On Mon, 20 Feb 2006, Liaw, Andy wrote:

> The last time I tried I didn't have much luck.  The gprof manual I could
> find seems to indicate that it can not profile code that are dynamically
> loaded.  (I was trying on Linux.)  The R source seems to hint otherwise.
> I'd very much appreciate pointers as well.
>

I happily use `oprofile' (web page at http://oprofile.sourceforge.net/)
for profiling shared libs. You simply start a daemon, run your R code,
and get the results. `opannotate' labels lines in C source files with the
corresponding systime, very very cute.

Best,

Torsten

> Andy
>
> From: Ross Boylan
>>
>> Does anyone have any advice about profiling C/C++ code in a package
>> under R?  Does R need to be built specially for this to work?
>>
>> The FAQ has some entries about profiling but they cover R level
>> profiling; I'm try to get at the C++ code I've written that is called
>> from R.
>>
>> Primary target is Mac OS X.
>>
>> Thanks.
>>
>> Ross Boylan
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: profiling C code

Brian Ripley
In reply to this post by Liaw, Andy
On Mon, 20 Feb 2006, Liaw, Andy wrote:

> The last time I tried I didn't have much luck.  The gprof manual I could
> find seems to indicate that it can not profile code that are dynamically
> loaded.  (I was trying on Linux.)  The R source seems to hint otherwise.

grof is not mentioned in any of the R manuals, so what are your referring
to?  (R-admin talks about `to compile a profiling version of R', not of
shared libraries.)

> I'd very much appreciate pointers as well.

You need to use sprof rather than gprof.  Here's one relevant thread:

http://tolstoy.newcastle.edu.au/~rking/R/devel/05/02/2351.html

and I am pretty sure there have been others.

Linux has very sketchy info on sprof.  However,

http://people.redhat.com/drepper/dsohowto.pdf

has some.  I've thought about putting something about this in the
R-exts manual


>
> Andy
>
> From: Ross Boylan
>>
>> Does anyone have any advice about profiling C/C++ code in a package
>> under R?  Does R need to be built specially for this to work?
>>
>> The FAQ has some entries about profiling but they cover R level
>> profiling; I'm try to get at the C++ code I've written that is called
>> from R.
>>
>> Primary target is Mac OS X.
>>
>> Thanks.
>>
>> Ross Boylan
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: profiling C code

Thomas Lumley
In reply to this post by Ross Boylan
On Mon, 20 Feb 2006, Ross Boylan wrote:

> Does anyone have any advice about profiling C/C++ code in a package
> under R?  Does R need to be built specially for this to work?
>
> The FAQ has some entries about profiling but they cover R level
> profiling; I'm try to get at the C++ code I've written that is called
> from R.
>
> Primary target is Mac OS X.

Under OS X I use 'sample', which doesn't require any recompiling (or
Sampler.app, which is a GUI version)

  -thomas

>
> Thanks.
>
> Ross Boylan
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

Thomas Lumley Assoc. Professor, Biostatistics
[hidden email] University of Washington, Seattle

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: profiling C code

Brian Ripley
In reply to this post by Brian Ripley
Some more information.

1) Whether gprof works is system-specific.  Solaris says in man gprof

   64-bit profiling
      64-bit profiling may be used freely with dynamically  linked
      executables,  and profiling information is collected for the
      shared objects if the objects are  compiled  for  profiling.

   32-bit profiling
      32-bit profiling may be used with dynamically linked execut-
      ables, but care must be applied. In 32-bit profiling, shared
      objects cannot be profiled with gprof.

As usual, Linux seems not to give such details (or if it does, please
point me at them).

2) Here is an example of using sprof.  I needed to be root to create
some directories:

% mkdir -p /var/tmp//users/ripley/R/R-devel/library/stats/libs
% chown -R ripley:bdr /var/tmp//users/ripley
% setenv LD_PROFILE /users/ripley/R/R-devel/library/stats/libs/stats.so

% R
> example(smooth.spline)
> q()

% sprof /users/ripley/R/R-devel/library/stats/libs/stats.so \
/var/tmp/users/ripley/R/R-devel/library/stats/libs/stats.so.profile

Flat profile:

Each sample counts as 0.01 seconds.
   %   cumulative   self              self     total
  time   seconds   seconds    calls  us/call  us/call  name
100.00      0.01     0.01     2914     3.43           bsplvb_
   0.00      0.01     0.00     2261     0.00           bvalue_
   0.00      0.01     0.00     2186     0.00           bsplvd_
   0.00      0.01     0.00       91     0.00           sinerp_
   0.00      0.01     0.00       91     0.00           sslvrg_
   0.00      0.01     0.00        9     0.00           sbart_
   0.00      0.01     0.00        9     0.00           sgram_
   0.00      0.01     0.00        9     0.00           stxwx_
...

If you are running Linux, Torsten's suggestion of oprofile is a good one,
provided you have root access (it seems to need it).  So I used in a root
terminal (running the same example in another window in the middle)

% opcontrol --no-vmlinux
% opcontrol --start
.... run things ...
% opreport -l /users/ripley/R/R-devel/bin/exec/R
% opreport -l /users/ripley/R/R-devel/library/stats/libs/stats.so
...
(No unit mask) count 100000
samples  %        symbol name
6        31.5789  bvalue_
4        21.0526  bsplvb_
3        15.7895  bsplvd_
2        10.5263  sbart_
2        10.5263  sslvrg_
1         5.2632  anonymous symbol from section .plt
1         5.2632  stxwx_
% opannotate -s /users/ripley/R/R-devel/library/stats/libs/stats.so
[Problem, root cannot read my source files]
% opcontrol --shutdown

If there is a way to use this without root access I would like to know it,
as only a few very privileged users get that here, and I am not going to
open up root access to my private file server.

Since it does look worthwhile collecting this sort of information together
for R-exts, please add further contributions (to the list or direct to
me).


On Tue, 21 Feb 2006, Prof Brian Ripley wrote:

> On Mon, 20 Feb 2006, Liaw, Andy wrote:
>
>> The last time I tried I didn't have much luck.  The gprof manual I could
>> find seems to indicate that it can not profile code that are dynamically
>> loaded.  (I was trying on Linux.)  The R source seems to hint otherwise.
>
> grof is not mentioned in any of the R manuals, so what are your referring
> to?  (R-admin talks about `to compile a profiling version of R', not of
> shared libraries.)
>
>> I'd very much appreciate pointers as well.
>
> You need to use sprof rather than gprof.  Here's one relevant thread:
>
> http://tolstoy.newcastle.edu.au/~rking/R/devel/05/02/2351.html
>
> and I am pretty sure there have been others.
>
> Linux has very sketchy info on sprof.  However,
>
> http://people.redhat.com/drepper/dsohowto.pdf
>
> has some.  I've thought about putting something about this in the
> R-exts manual
>
>
>>
>> Andy
>>
>> From: Ross Boylan
>>>
>>> Does anyone have any advice about profiling C/C++ code in a package
>>> under R?  Does R need to be built specially for this to work?
>>>
>>> The FAQ has some entries about profiling but they cover R level
>>> profiling; I'm try to get at the C++ code I've written that is called
>>> from R.
>>>
>>> Primary target is Mac OS X.
>>>
>>> Thanks.
>>>
>>> Ross Boylan
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>>
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>
>

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: profiling C code

Simon Urbanek
In reply to this post by Ross Boylan

On Feb 20, 2006, at 7:18 PM, Ross Boylan wrote:

> Does anyone have any advice about profiling C/C++ code in a package  
> under R?  Does R need to be built specially for this to work?
>
> The FAQ has some entries about profiling but they cover R level  
> profiling; I'm try to get at the C++ code I've written that is  
> called from R.
>
> Primary target is Mac OS X.
>

If you have an OS X box to work on, then you have a handful of really  
excellent tools. Have a look at Shark from the CHUD tools (in /
Developer/Applications/Performance Tools) Although it's sampling-
based (.i.e. no need to gprof - just run sample running R), it is  
remarkably accurate and has ton of features for what they call 'data  
mining' of the profile data - including drill-down to source code and  
even assembly level. You can read a bit at
http://developer.apple.com/tools/sharkoptimize.html
It proves to be really useful and very flexible.

Cheers,
Simon

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: profiling C code

Torsten Hothorn
In reply to this post by Brian Ripley

>
> If you are running Linux, Torsten's suggestion of oprofile is a good one,
> provided you have root access (it seems to need it).  So I used in a root
> terminal (running the same example in another window in the middle)
>
> % opcontrol --no-vmlinux
> % opcontrol --start
> .... run things ...

opcontrol --dump

doesn't require root privileges on my system and

> % opreport -l /users/ripley/R/R-devel/bin/exec/R
> % opreport -l /users/ripley/R/R-devel/library/stats/libs/stats.so

should be allowed for `normal users', too.

> ...
> (No unit mask) count 100000
> samples  %        symbol name
> 6        31.5789  bvalue_
> 4        21.0526  bsplvb_
> 3        15.7895  bsplvd_
> 2        10.5263  sbart_
> 2        10.5263  sslvrg_
> 1         5.2632  anonymous symbol from section .plt
> 1         5.2632  stxwx_
> % opannotate -s /users/ripley/R/R-devel/library/stats/libs/stats.so
> [Problem, root cannot read my source files]

for me, opannotate did not ask about running as `root'.

> % opcontrol --shutdown
>
> If there is a way to use this without root access I would like to know it,
> as only a few very privileged users get that here, and I am not going to
> open up root access to my private file server.

root needs to start the daemon via

opcontrol --start-daemon

and normal users can dump the output. But I don't know how a normal user
would clean up and start new.

Best,

Torsten

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: profiling C code

Brian Ripley
On Wed, 22 Feb 2006, Torsten Hothorn wrote:

>
>>
>> If you are running Linux, Torsten's suggestion of oprofile is a good one,
>> provided you have root access (it seems to need it).  So I used in a root
>> terminal (running the same example in another window in the middle)
>>
>> % opcontrol --no-vmlinux
>> % opcontrol --start
>> .... run things ...
>
> opcontrol --dump
>
> doesn't require root privileges on my system and

Yes, true, but not much use unless you can start a daemon.

>> % opreport -l /users/ripley/R/R-devel/bin/exec/R
>> % opreport -l /users/ripley/R/R-devel/library/stats/libs/stats.so
>
> should be allowed for `normal users', too.

Interesting.  It seems to be allowed unless the daemon is running and has
undumped data on the file.  So the advice seems to need to be to dump and
then run opreport.

>> ...
>> (No unit mask) count 100000
>> samples  %        symbol name
>> 6        31.5789  bvalue_
>> 4        21.0526  bsplvb_
>> 3        15.7895  bsplvd_
>> 2        10.5263  sbart_
>> 2        10.5263  sslvrg_
>> 1         5.2632  anonymous symbol from section .plt
>> 1         5.2632  stxwx_
>> % opannotate -s /users/ripley/R/R-devel/library/stats/libs/stats.so
>> [Problem, root cannot read my source files]
>
> for me, opannotate did not ask about running as `root'.

The issue is that that my source files are on a private file server to
which the root account has no access.  That should be common for any
secure system with remote-mounted file systems.  However, once I dump the
data I can run opannotate as a normal user.

>
>> % opcontrol --shutdown
>>
>> If there is a way to use this without root access I would like to know it,
>> as only a few very privileged users get that here, and I am not going to
>> open up root access to my private file server.
>
> root needs to start the daemon via
>
> opcontrol --start-daemon
>
> and normal users can dump the output. But I don't know how a normal user
> would clean up and start new.

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel