Linux distribution with gcc 4.8 and AddressSanitizer ?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Linux distribution with gcc 4.8 and AddressSanitizer ?

Thomas Petzoldt
Dear R developers,

I've got an information from Prof. Ripley regarding a bug found with
AdressSanitizer in one of our packages. It is now fixed, thank you for
this information.

Now, I would like to run AddressSanitizer myself before submitting the
patched package to CRAN.

Is there a recommendation of a suitable Linux distribution with gcc 4.8,
ideally an ISO image or (even better) a virtual appliance for VMware or
VirtalBox? My Debian Wheezy machines have only 4.7.2.

Thank you

Thomas Petzoldt


--
Dr. Thomas Petzoldt
Technische Universitaet Dresden
Faculty of Environmental Sciences
Institute of Hydrobiology
01062 Dresden, Germany

E-Mail: [hidden email]
http://tu-dresden.de/Members/thomas.petzoldt

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Linux distribution with gcc 4.8 and AddressSanitizer ?

José Matos-2
On Thursday 18 April 2013 17:38:06 Thomas Petzoldt wrote:

> Dear R developers,
>
> I've got an information from Prof. Ripley regarding a bug found with
> AdressSanitizer in one of our packages. It is now fixed, thank you for
> this information.
>
> Now, I would like to run AddressSanitizer myself before submitting the
> patched package to CRAN.
>
> Is there a recommendation of a suitable Linux distribution with gcc 4.8,
> ideally an ISO image or (even better) a virtual appliance for VMware or
> VirtalBox? My Debian Wheezy machines have only 4.7.2.
>
> Thank you
>
> Thomas Petzoldt

I am not sure about all the requisites above (regarding the virtual appliances although I know that they are available) but Fedora 19 (Alpha) that will be released today has gcc 4.8.

Even although it has the Alpha moniker, and the corresponding stage, it is relatively stable and thus suitable for your requirements.

Regards,
--
José Abílio

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Linux distribution with gcc 4.8 and AddressSanitizer ?

Prof Brian Ripley
In reply to this post by Thomas Petzoldt
On 18/04/2013 16:38, Thomas Petzoldt wrote:

> Dear R developers,
>
> I've got an information from Prof. Ripley regarding a bug found with
> AddressSanitizer in one of our packages. It is now fixed, thank you for
> this information.
>
> Now, I would like to run AddressSanitizer myself before submitting the
> patched package to CRAN.
>
> Is there a recommendation of a suitable Linux distribution with gcc 4.8,
> ideally an ISO image or (even better) a virtual appliance for VMware or
> VirtalBox? My Debian Wheezy machines have only 4.7.2.
>
> Thank you
>
> Thomas Petzoldt
>

gcc 4.8.0 has only been released for 4 weeks, so I doubt any released
distro has it.  Fedora Rawhide (19-to-be) has it, and so I am told does
Debian experimental.

I compiled gcc myself from the sources, but my sysadmins offered to make
a Rawhide virtual machine or live DVD for me.

For those who do not know what AddressSanitizer is, see
http://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Using-gctorture-and-memory-access 
.

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Linux distribution with gcc 4.8 and AddressSanitizer ?

Dirk Eddelbuettel

On 18 April 2013 at 17:19, Prof Brian Ripley wrote:
| On 18/04/2013 16:38, Thomas Petzoldt wrote:
| > Dear R developers,
| >
| > I've got an information from Prof. Ripley regarding a bug found with
| > AddressSanitizer in one of our packages. It is now fixed, thank you for
| > this information.
| >
| > Now, I would like to run AddressSanitizer myself before submitting the
| > patched package to CRAN.
| >
| > Is there a recommendation of a suitable Linux distribution with gcc 4.8,
| > ideally an ISO image or (even better) a virtual appliance for VMware or
| > VirtalBox? My Debian Wheezy machines have only 4.7.2.
| >
| > Thank you
| >
| > Thomas Petzoldt
| >
|
| gcc 4.8.0 has only been released for 4 weeks, so I doubt any released
| distro has it.  Fedora Rawhide (19-to-be) has it, and so I am told does
| Debian experimental.

Yes, looking at http://packages.debian.org/experimental/gcc-4.8 reveals that
there were build -1 on March 22 and -2 on March 28 --- but also fifteen (15)
prerelease builds going back all the way to Nov 2012.
 
| I compiled gcc myself from the sources, but my sysadmins offered to make
| a Rawhide virtual machine or live DVD for me.

apt-get could you your friend too.  And virtual machines are pretty easy,
like the virtd / kvm variant (on Linux) quite a bit.

Dirk

--
Dirk Eddelbuettel | [hidden email] | http://dirk.eddelbuettel.com

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Linux distribution with gcc 4.8 and AddressSanitizer ?

Thomas Petzoldt
In reply to this post by José Matos-2
On 18.04.2013 18:05, José Matos wrote:

> On Thursday 18 April 2013 17:38:06 Thomas Petzoldt wrote:
>> Dear R developers,
>>
>> I've got an information from Prof. Ripley regarding a bug found
>> with AdressSanitizer in one of our packages. It is now fixed, thank
>> you for this information.
>>
>> Now, I would like to run AddressSanitizer myself before submitting
>> the patched package to CRAN.
>>
>> Is there a recommendation of a suitable Linux distribution with gcc
>> 4.8, ideally an ISO image or (even better) a virtual appliance for
>> VMware or VirtalBox? My Debian Wheezy machines have only 4.7.2.
>>
>> Thank you
>>
>> Thomas Petzoldt
>
> I am not sure about all the requisites above (regarding the virtual
> appliances although I know that they are available) but Fedora 19
> (Alpha) that will be released today has gcc 4.8.
>
> Even although it has the Alpha moniker, and the corresponding stage,
> it is relatively stable and thus suitable for your requirements.
>
> Regards,
>

Thank you for the hint to use Fedora 19 Alpha. I have it now running,
together with R 3.0.0. and gcc 4.8.0 20120412 (Red Hat 4.8.0-2).

Compilation and installation of packages (without ASAN) workes out of
the box.

Then I've set:

export PKG_CFLAGS="-fsanitize=address -fno-omit-frame-pointer"

... and compilation runs and I see that gcc uses the flags, but package
installation still fails:

** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
   unable to load shared object
'/home/user/packages/deSolve.Rcheck/deSolve/libs/deSolve.so':
   /home/user/packages/deSolve.Rcheck/deSolve/libs/deSolve.so: undefined
symbol: __asan_report_load8
Error: loading failed
Execution halted
ERROR: loading failed


I see that the address sanitizer cannot work yet (__asan_report_load8)
and that I missed something important, but what?

Thomas Petzoldt



--
Thomas Petzoldt
Technische Universitaet Dresden
Faculty of Environmental Sciences
Institute of Hydrobiology
01062 Dresden, Germany

E-Mail: [hidden email]
http://tu-dresden.de/Members/thomas.petzoldt

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Linux distribution with gcc 4.8 and AddressSanitizer ?

Prof Brian Ripley
On 19/04/2013 18:22, Thomas Petzoldt wrote:

> On 18.04.2013 18:05, José Matos wrote:
>> On Thursday 18 April 2013 17:38:06 Thomas Petzoldt wrote:
>>> Dear R developers,
>>>
>>> I've got an information from Prof. Ripley regarding a bug found
>>> with AdressSanitizer in one of our packages. It is now fixed, thank
>>> you for this information.
>>>
>>> Now, I would like to run AddressSanitizer myself before submitting
>>> the patched package to CRAN.
>>>
>>> Is there a recommendation of a suitable Linux distribution with gcc
>>> 4.8, ideally an ISO image or (even better) a virtual appliance for
>>> VMware or VirtalBox? My Debian Wheezy machines have only 4.7.2.
>>>
>>> Thank you
>>>
>>> Thomas Petzoldt
>>
>> I am not sure about all the requisites above (regarding the virtual
>> appliances although I know that they are available) but Fedora 19
>> (Alpha) that will be released today has gcc 4.8.
>>
>> Even although it has the Alpha moniker, and the corresponding stage,
>> it is relatively stable and thus suitable for your requirements.
>>
>> Regards,
>>
>
> Thank you for the hint to use Fedora 19 Alpha. I have it now running,
> together with R 3.0.0. and gcc 4.8.0 20120412 (Red Hat 4.8.0-2).
>
> Compilation and installation of packages (without ASAN) workes out of
> the box.
>
> Then I've set:
>
> export PKG_CFLAGS="-fsanitize=address -fno-omit-frame-pointer"
>
> ... and compilation runs and I see that gcc uses the flags, but package
> installation still fails:
>
> ** testing if installed package can be loaded
> Error in dyn.load(file, DLLpath = DLLpath, ...) :
>    unable to load shared object
> '/home/user/packages/deSolve.Rcheck/deSolve/libs/deSolve.so':
>    /home/user/packages/deSolve.Rcheck/deSolve/libs/deSolve.so: undefined
> symbol: __asan_report_load8
> Error: loading failed
> Execution halted
> ERROR: loading failed
>
>
> I see that the address sanitizer cannot work yet (__asan_report_load8)
> and that I missed something important, but what?

The manual said

'It requires code to have been compiled and linked with -fsanitize=address'

I don't see that you linked with it ...

I have (in ~/.R/Makevars)

CC="gcc -std=gnu99 -fsanitize=address -fno-omit-frame-pointer"

and similar for F77, CXX ....

I guess PKG_LIBS=-fsanitize=address might work.



'
>
> Thomas Petzoldt
>
>
>


--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Linux distribution with gcc 4.8 and AddressSanitizer ?

Simon Urbanek

On Apr 19, 2013, at 2:19 PM, Prof Brian Ripley wrote:

> On 19/04/2013 18:22, Thomas Petzoldt wrote:
>> On 18.04.2013 18:05, José Matos wrote:
>>> On Thursday 18 April 2013 17:38:06 Thomas Petzoldt wrote:
>>>> Dear R developers,
>>>>
>>>> I've got an information from Prof. Ripley regarding a bug found
>>>> with AdressSanitizer in one of our packages. It is now fixed, thank
>>>> you for this information.
>>>>
>>>> Now, I would like to run AddressSanitizer myself before submitting
>>>> the patched package to CRAN.
>>>>
>>>> Is there a recommendation of a suitable Linux distribution with gcc
>>>> 4.8, ideally an ISO image or (even better) a virtual appliance for
>>>> VMware or VirtalBox? My Debian Wheezy machines have only 4.7.2.
>>>>
>>>> Thank you
>>>>
>>>> Thomas Petzoldt
>>>
>>> I am not sure about all the requisites above (regarding the virtual
>>> appliances although I know that they are available) but Fedora 19
>>> (Alpha) that will be released today has gcc 4.8.
>>>
>>> Even although it has the Alpha moniker, and the corresponding stage,
>>> it is relatively stable and thus suitable for your requirements.
>>>
>>> Regards,
>>>
>>
>> Thank you for the hint to use Fedora 19 Alpha. I have it now running,
>> together with R 3.0.0. and gcc 4.8.0 20120412 (Red Hat 4.8.0-2).
>>
>> Compilation and installation of packages (without ASAN) workes out of
>> the box.
>>
>> Then I've set:
>>
>> export PKG_CFLAGS="-fsanitize=address -fno-omit-frame-pointer"
>>
>> ... and compilation runs and I see that gcc uses the flags, but package
>> installation still fails:
>>
>> ** testing if installed package can be loaded
>> Error in dyn.load(file, DLLpath = DLLpath, ...) :
>>   unable to load shared object
>> '/home/user/packages/deSolve.Rcheck/deSolve/libs/deSolve.so':
>>   /home/user/packages/deSolve.Rcheck/deSolve/libs/deSolve.so: undefined
>> symbol: __asan_report_load8
>> Error: loading failed
>> Execution halted
>> ERROR: loading failed
>>
>>
>> I see that the address sanitizer cannot work yet (__asan_report_load8)
>> and that I missed something important, but what?
>
> The manual said
>
> 'It requires code to have been compiled and linked with -fsanitize=address'
>
> I don't see that you linked with it ...
>
> I have (in ~/.R/Makevars)
>
> CC="gcc -std=gnu99 -fsanitize=address -fno-omit-frame-pointer"
>
> and similar for F77, CXX ....
>
> I guess PKG_LIBS=-fsanitize=address might work.
>

Just a data point: that didn't work for me on OS X using clang. I had to compile R with clang + asan, because clang seems to embed the run-time only in executables, not shared objects and when linking the .so to the runtime manually it broke. But building R with clang 3.3 from trunk and gfortran from CRAN did work with some tweaking - this is what I used:
CC=/opt/clang/bin/clang CXX=/opt/clang/bin/clang++ 'CFLAGS=-fsanitize=address -O1 -fno-omit-frame-pointer -g -Wall' 'CXXFLAGS=-fsanitize=address -O1 -fno-omit-frame-pointer -g -Wall' 'LDFLAGS=-fsanitize=address -L/usr/local/lib -lgfortran' 'FC=gfortran-4.2 -m64' 'F77=gfortran-4.2 -m64'

Another data point is that R segfaulted on start for me when compiled with gcc 4.8.0-3 from Debian experimental (amd64) with asan enabled (and I gave up when I couldn't get gfortran-4.8 to mix with clang-3.2 to try that route). Those are all very fragile tools, so it's not unexpected, but essentially it seems that it's not trivial to get a working set of tools...

Cheers,
Simon



>
>
> '
>>
>> Thomas Petzoldt
>>
>>
>>
>
>
> --
> Brian D. Ripley,                  [hidden email]
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Linux distribution with gcc 4.8 and AddressSanitizer -- solved

Thomas Petzoldt
In reply to this post by José Matos-2
It works!

After some hours of compilation, reading the docs and testing,
I got it now working and was able to reproduce (and fix) the
reported error message.

Then ingredients of the successful AdressSanitizer (ASAN)
system were:

- Fedora 19 Alpha RC4 with gcc 4.8 on VirtualBox,
- manual installation of several additional libraries
   especially libasan-devel,
- setting of Makevars and a few environment variables,
- compilation of R-devel (2013-04-19) with address-sanitizer
   (and --enable-strict barrier)
   ==> the compilation of R itself went through
       without problems so that R runs without crash.

Finally:
- compilation and ASAN check of the affected package
   that reproduced the error message.
- bugfix and successful final test.


Maybe this was not the most parsimonious approach ;-)
but using a suitable self-compiled R seems to be unavoidable.



Again, many thanks for your help and the great R system!

Thomas P.



--
Thomas Petzoldt
Technische Universitaet Dresden
Faculty of Environmental Sciences
Institute of Hydrobiology
01062 Dresden, Germany

E-Mail: [hidden email]
http://tu-dresden.de/Members/thomas.petzoldt

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel