difficulty with sanitizer using bigmemory

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

difficulty with sanitizer using bigmemory

petem
Dear all,

bigKRLS, which has been on CRAN for a couple of years, had to be pulled
recently due to what seems to be a sanitizer issue stemming from its use of
bigmemory. bigKRLS works fine (we’ve used it ourselves on many different
platforms and have had over 15,000 downloads without an end user reporting
difficulties because of this issue). Unfortunately, we have been unable to
reproduce the UndefinedBehaviorSanitizer error locally, and are unsure how
to proceed. We corresponded with Michael Kane, author of bigmemory, who
seems to think the issue may be on the CRAN side. At his suggestion, we
switched to filebacked big matrices, which didn’t resolve the issue. We
also tried adding MAIN_LDFLAGS = -fsanitize=undefined,address to the
MAKEVARS, which didn’t do the trick either.

Here are some links:


   1.

   bigKRLS repo: https://github.com/rdrr1990/bigKRLS
   2.

   bigKRLS file backs all big matrices with a helper function called
   ‘to.big.matrix’ found starting on line 116 of
   bigKRLS/bigKRLS_Rcpp_functions.R:
   https://github.com/rdrr1990/bigKRLS/blob/master/R/bigKRLS_Rcpp_functions.R
   3.

   Discussion with Michael Kane:
   https://github.com/kaneplusplus/bigmemory/issues/73#issuecomment-330543207



And here is the main error itself:

package bigKRLS_3.0.5.tar.gz has been auto-processed.

The auto-check found additional issues for the *last* version released on
CRAN:

  clang-UBSAN <
https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-UBSAN/bigKRLS>

  gcc-UBSAN <https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-UBSAN/bigKRLS>

Clicking through, you find that

> test_check("bigKRLS")

gauss_kernel.cpp:38:40: runtime error: member call on address
0x6120001f2c40 which does not point to an object of type 'BigMatrix'

0x6120001f2c40: note: object is of type 'FileBackedBigMatrix'

[details omitted]

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
gauss_kernel.cpp:38:40 in

/data/gannet/ripley/R/test-clang/bigmemory/include/bigmemory/BigMatrix.h:41:28:
runtime error: member access within address 0x6120001f2c40 which does not
point to an object of type 'BigMatrix'

0x6120001f2c40: note: object is of type 'FileBackedBigMatrix'

But FileBackedBigMatrix are BigMatrix objects and earlier versions of the
package which did not file back generated similar errors. (R CMD check
--as-cran does not yield these errors on Ubuntu 16.04 or MacOS Sierra,
which has made it particularly tricky to work on.)

Any and all help would be greatly appreciated!!

Best,

Pete Mohanty and Robert Shaffer


--
Pete Mohanty, PhD
Science, Engineering & Education Fellow
Stanford University
Department of Statistics
https://sites.google.com/site/petemohanty/

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: difficulty with sanitizer using bigmemory

Henrik Bengtsson-5
The https://builder.r-hub.io/advanced provides an 'R CMD check
--as-cran' for 'Debian Linux, R-devel, GCC ASAN/UBSAN'.  You can
either upload it there or launch it from you local R prompt;

> rhub::check()
── Choose build platform
...
10: Debian Linux, R-devel, GCC ASAN/UBSAN (linux-x86_64-rocker-gcc-san)
...
20: Windows Server 2008 R2 SP1, R-release, 32/64 bit (windows-x86_64-release)

With some luck, you might be able to reproduce those CRAN sanity checks there.

/Henrik

On Wed, Jul 24, 2019 at 9:21 AM pete mohanty <[hidden email]> wrote:

>
> Dear all,
>
> bigKRLS, which has been on CRAN for a couple of years, had to be pulled
> recently due to what seems to be a sanitizer issue stemming from its use of
> bigmemory. bigKRLS works fine (we’ve used it ourselves on many different
> platforms and have had over 15,000 downloads without an end user reporting
> difficulties because of this issue). Unfortunately, we have been unable to
> reproduce the UndefinedBehaviorSanitizer error locally, and are unsure how
> to proceed. We corresponded with Michael Kane, author of bigmemory, who
> seems to think the issue may be on the CRAN side. At his suggestion, we
> switched to filebacked big matrices, which didn’t resolve the issue. We
> also tried adding MAIN_LDFLAGS = -fsanitize=undefined,address to the
> MAKEVARS, which didn’t do the trick either.
>
> Here are some links:
>
>
>    1.
>
>    bigKRLS repo: https://github.com/rdrr1990/bigKRLS
>    2.
>
>    bigKRLS file backs all big matrices with a helper function called
>    ‘to.big.matrix’ found starting on line 116 of
>    bigKRLS/bigKRLS_Rcpp_functions.R:
>    https://github.com/rdrr1990/bigKRLS/blob/master/R/bigKRLS_Rcpp_functions.R
>    3.
>
>    Discussion with Michael Kane:
>    https://github.com/kaneplusplus/bigmemory/issues/73#issuecomment-330543207
>
>
>
> And here is the main error itself:
>
> package bigKRLS_3.0.5.tar.gz has been auto-processed.
>
> The auto-check found additional issues for the *last* version released on
> CRAN:
>
>   clang-UBSAN <
> https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-UBSAN/bigKRLS>
>
>   gcc-UBSAN <https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-UBSAN/bigKRLS>
>
> Clicking through, you find that
>
> > test_check("bigKRLS")
>
> gauss_kernel.cpp:38:40: runtime error: member call on address
> 0x6120001f2c40 which does not point to an object of type 'BigMatrix'
>
> 0x6120001f2c40: note: object is of type 'FileBackedBigMatrix'
>
> [details omitted]
>
> SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
> gauss_kernel.cpp:38:40 in
>
> /data/gannet/ripley/R/test-clang/bigmemory/include/bigmemory/BigMatrix.h:41:28:
> runtime error: member access within address 0x6120001f2c40 which does not
> point to an object of type 'BigMatrix'
>
> 0x6120001f2c40: note: object is of type 'FileBackedBigMatrix'
>
> But FileBackedBigMatrix are BigMatrix objects and earlier versions of the
> package which did not file back generated similar errors. (R CMD check
> --as-cran does not yield these errors on Ubuntu 16.04 or MacOS Sierra,
> which has made it particularly tricky to work on.)
>
> Any and all help would be greatly appreciated!!
>
> Best,
>
> Pete Mohanty and Robert Shaffer
>
>
> --
> Pete Mohanty, PhD
> Science, Engineering & Education Fellow
> Stanford University
> Department of Statistics
> https://sites.google.com/site/petemohanty/
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel