R 4.0.1-4.0.2 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

R 4.0.1-4.0.2 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7

Ryan Novosielski
Hi there,

I initially asked about this on r-help and was told this might be a better venue. I’m not really convinced from reading the posting guide, but I’ll give it a shot. It was also suggested that the R-Project doesn’t really care about building with “non-standard” compilers, but I can’t find any evidence of that on the website (indeed, there’s some mention of successful past builds, and the build itself is successful).

I built R 4.0.2 with the Intel Parallel Studio XE compiler suite, versions 19.0.x to 19.1.1. Build seems to go fine. I built it like this:

module purge
module load intel/19.1.1
module list

export CC=icc
export CXX=icpc
export F77=ifort
export FC=ifort
export AR=xiar
export LD=xild

export CFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
export F77FLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
export FFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
export CXXFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
export MKL="-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread"

VERSION=4.0.1

/scratch/novosirj/install-files/R-${VERSION}/configure --with-blas="$MKL" --with-lapack --prefix=/opt/sw/packages/intel-19_1/R-Project/${VERSION} && \
       make -j32 && make check && make -j32 install

However, the “make check" phase fails at this part:

Testing examples for package ‘parallel’
make[2]: Leaving directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests/Examples'
make[1]: Leaving directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
make[1]: Entering directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
running strict specific tests
make[2]: Entering directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
running code in '/scratch/novosirj/install-files/R-4.0.1/tests/eval-etc.R' ... OK
 comparing 'eval-etc.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/eval-etc.Rout.save' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.1/tests/simple-true.R' ... OK
 comparing 'simple-true.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/simple-true.Rout.save' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.1/tests/arith-true.R' ... OK
 comparing 'arith-true.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/arith-true.Rout.save' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.1/tests/arith.R' ... OK
 comparing 'arith.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/arith.Rout.save' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.1/tests/lm-tests.R' ... OK
 comparing 'lm-tests.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/lm-tests.Rout.save' ... OK
/bin/sh: line 1: 62064 Segmentation fault      (core dumped) LANGUAGE=en LC_ALL=C SRCDIR=/scratch/novosirj/install-files/R-4.0.1/tests R_DEFAULT_PACKAGES= ../bin/R --vanilla < /scratch/novosirj/install-files/R-4.0.1/tests/ok-errors.R > ok-errors.Rout.fail 2>&1
running code in '/scratch/novosirj/install-files/R-4.0.1/tests/ok-errors.R' ...make[2]: *** [ok-errors.Rout] Error 1
make[2]: Leaving directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
make[1]: *** [test-Specific] Error 2
make[1]: Leaving directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
make: *** [test-all-basics] Error 1

Is this something I should be concerned about, or something I can fix? Not seeing any real information about what’s going wrong here. Here’s what’s contained in ok-errors.Rout.fail:

---
R version 4.0.1 (2020-06-06) -- "See Things Now"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> #### STRICT test suite in the spirit of no-segfaults,
> #### but with explicit statements.
>
> options(error=expression(NULL))
> stop("test of `options(error=expression(NULL))'")
Error: test of `options(error=expression(NULL))'
>
> if(FALSE) {
+ ## these ought to work on machines with enough memory
+ ## These segfaulted in 1.3.x ,  give "could not allocate" errors now
+   integer(2^30+1)
+    double(2^30+1)
+   complex(2^30+1)
+ character(2^30+1)
+ vector("list", 2^30+2)
+ }
>
> ## bad infinite recursion / on.exit / ... interactions
> ##   catch the error to permit different error messages emitted
> ##   (handling of infinite recursion is different in the AST interpreter
> ##   and the byte-code interpreter)
>
> bar <- function() 1+1
> foo <- function() { on.exit(bar()); foo() }
> tryCatch(foo(), error=function(x) TRUE) # now simple "infinite recursion"

*** caught segfault ***
address 0x7fff4dc1b9f8, cause 'memory not mapped'

Traceback:
1: foo()
2: foo()
3: foo()
4: foo()

...

2712: foo()
2713: foo()
2714: foo()
2715: foo()
2716: foo()
2717: foo()
2718: foo()
2719: doTryCatch(return(expr), name, parentenv, handler)
2720: tryCatchOne(expr, names, parentenv, handlers[[1L]])
2721: tryCatchList(expr, classes, parentenv, handlers)
2722: tryCatch(foo(), error = function(x) TRUE)
An irrecoverable exception occurred. R is aborting now ...
---

Thanks in advance.

--
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
    `'


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.2 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7

Duncan Murdoch-2
If you just run the failed test, does it segfault?  What I get when I
run it is

 > bar <- function() 1+1
 > foo <- function() { on.exit(bar()); foo() }
 > tryCatch(foo(), error=function(x) TRUE) # now simple "infinite recursion"
[1] TRUE

Clearly a segfault on an infinite recursion is undesirable, but maybe
not the end of the world: maybe it just makes your life harder when
debugging.  Or maybe you'll get segfaults from non-buggy code too, which
would be really bad.  But I think you're the only one who can debug
this, and find out why R's error handling isn't working in your build.

Duncan Murdoch

On 24/06/2020 2:56 p.m., Ryan Novosielski wrote:

> Hi there,
>
> I initially asked about this on r-help and was told this might be a better venue. I’m not really convinced from reading the posting guide, but I’ll give it a shot. It was also suggested that the R-Project doesn’t really care about building with “non-standard” compilers, but I can’t find any evidence of that on the website (indeed, there’s some mention of successful past builds, and the build itself is successful).
>
> I built R 4.0.2 with the Intel Parallel Studio XE compiler suite, versions 19.0.x to 19.1.1. Build seems to go fine. I built it like this:
>
> module purge
> module load intel/19.1.1
> module list
>
> export CC=icc
> export CXX=icpc
> export F77=ifort
> export FC=ifort
> export AR=xiar
> export LD=xild
>
> export CFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> export F77FLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> export FFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> export CXXFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> export MKL="-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread"
>
> VERSION=4.0.1
>
> /scratch/novosirj/install-files/R-${VERSION}/configure --with-blas="$MKL" --with-lapack --prefix=/opt/sw/packages/intel-19_1/R-Project/${VERSION} && \
>         make -j32 && make check && make -j32 install
>
> However, the “make check" phase fails at this part:
>
> Testing examples for package ‘parallel’
> make[2]: Leaving directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests/Examples'
> make[1]: Leaving directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
> make[1]: Entering directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
> running strict specific tests
> make[2]: Entering directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
> running code in '/scratch/novosirj/install-files/R-4.0.1/tests/eval-etc.R' ... OK
>   comparing 'eval-etc.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/eval-etc.Rout.save' ... OK
> running code in '/scratch/novosirj/install-files/R-4.0.1/tests/simple-true.R' ... OK
>   comparing 'simple-true.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/simple-true.Rout.save' ... OK
> running code in '/scratch/novosirj/install-files/R-4.0.1/tests/arith-true.R' ... OK
>   comparing 'arith-true.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/arith-true.Rout.save' ... OK
> running code in '/scratch/novosirj/install-files/R-4.0.1/tests/arith.R' ... OK
>   comparing 'arith.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/arith.Rout.save' ... OK
> running code in '/scratch/novosirj/install-files/R-4.0.1/tests/lm-tests.R' ... OK
>   comparing 'lm-tests.Rout' to '/scratch/novosirj/install-files/R-4.0.1/tests/lm-tests.Rout.save' ... OK
> /bin/sh: line 1: 62064 Segmentation fault      (core dumped) LANGUAGE=en LC_ALL=C SRCDIR=/scratch/novosirj/install-files/R-4.0.1/tests R_DEFAULT_PACKAGES= ../bin/R --vanilla < /scratch/novosirj/install-files/R-4.0.1/tests/ok-errors.R > ok-errors.Rout.fail 2>&1
> running code in '/scratch/novosirj/install-files/R-4.0.1/tests/ok-errors.R' ...make[2]: *** [ok-errors.Rout] Error 1
> make[2]: Leaving directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
> make[1]: *** [test-Specific] Error 2
> make[1]: Leaving directory `/mnt/scratch/novosirj/R-4.0.1-intel-19.1-build/tests'
> make: *** [test-all-basics] Error 1
>
> Is this something I should be concerned about, or something I can fix? Not seeing any real information about what’s going wrong here. Here’s what’s contained in ok-errors.Rout.fail:
>
> ---
> R version 4.0.1 (2020-06-06) -- "See Things Now"
> Copyright (C) 2020 The R Foundation for Statistical Computing
> Platform: x86_64-pc-linux-gnu (64-bit)
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under certain conditions.
> Type 'license()' or 'licence()' for distribution details.
>
> R is a collaborative project with many contributors.
> Type 'contributors()' for more information and
> 'citation()' on how to cite R or R packages in publications.
>
> Type 'demo()' for some demos, 'help()' for on-line help, or
> 'help.start()' for an HTML browser interface to help.
> Type 'q()' to quit R.
>
>> #### STRICT test suite in the spirit of no-segfaults,
>> #### but with explicit statements.
>>
>> options(error=expression(NULL))
>> stop("test of `options(error=expression(NULL))'")
> Error: test of `options(error=expression(NULL))'
>>
>> if(FALSE) {
> + ## these ought to work on machines with enough memory
> + ## These segfaulted in 1.3.x ,  give "could not allocate" errors now
> +   integer(2^30+1)
> +    double(2^30+1)
> +   complex(2^30+1)
> + character(2^30+1)
> + vector("list", 2^30+2)
> + }
>>
>> ## bad infinite recursion / on.exit / ... interactions
>> ##   catch the error to permit different error messages emitted
>> ##   (handling of infinite recursion is different in the AST interpreter
>> ##   and the byte-code interpreter)
>>
>> bar <- function() 1+1
>> foo <- function() { on.exit(bar()); foo() }
>> tryCatch(foo(), error=function(x) TRUE) # now simple "infinite recursion"
>
> *** caught segfault ***
> address 0x7fff4dc1b9f8, cause 'memory not mapped'
>
> Traceback:
> 1: foo()
> 2: foo()
> 3: foo()
> 4: foo()
>
> ...
>
> 2712: foo()
> 2713: foo()
> 2714: foo()
> 2715: foo()
> 2716: foo()
> 2717: foo()
> 2718: foo()
> 2719: doTryCatch(return(expr), name, parentenv, handler)
> 2720: tryCatchOne(expr, names, parentenv, handlers[[1L]])
> 2721: tryCatchList(expr, classes, parentenv, handlers)
> 2722: tryCatch(foo(), error = function(x) TRUE)
> An irrecoverable exception occurred. R is aborting now ...
> ---
>
> Thanks in advance.
>
> --
> ____
> || \\UTGERS,   |---------------------------*O*---------------------------
> ||_// the State |         Ryan Novosielski - [hidden email]
> || \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
> ||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
>      `'
>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.2 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7

Ivan Krylov
In reply to this post by Ryan Novosielski
On Wed, 24 Jun 2020 18:56:06 +0000
Ryan Novosielski <[hidden email]> wrote:

On my machine, getOption('expressions') is 5000 and the example from
the test correctly stops with length(traceback()) == 2500. (And the
simpler example of f <- function() f(); f() stops with
length(traceback()) == 5000).

> Traceback:

<...>

> 2718: foo()

This (traceback() being more than 2500 entries long) seems to imply
that the stack size check is somehow skipped. (Perhaps optimized away?)
The evaluation depth limit is checked in src/main/eval.c, line 705 [*],
followed by stack size check. Can you attach the debugger and take a
look at the values of R_EvalDepth and R_Expressions while executing the
text? What about R_CStackStart and R_CStackLimit? What is the stack
size limit (ulimit -s?) on the machine running this test?

--
Best regards,
Ivan

[*]
https://github.com/wch/r-source/blob/8d7ac4699fba640d030703fa010b66bf26054cbd/src/main/eval.c#L705

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

attachment0 (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.2 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7

Bjørn-Helge Mevik-3
In reply to this post by Ryan Novosielski
Ryan Novosielski <[hidden email]> writes:

> Hi there,
>
> I initially asked about this on r-help and was told this might be a better venue. I’m not really convinced from reading the posting guide, but I’ll give it a shot. It was also suggested that the R-Project doesn’t really care about building with “non-standard” compilers, but I can’t find any evidence of that on the website (indeed, there’s some mention of successful past builds, and the build itself is successful).
>
> I built R 4.0.2 with the Intel Parallel Studio XE compiler suite, versions 19.0.x to 19.1.1. Build seems to go fine. I built it like this:
>
> module purge
> module load intel/19.1.1
> module list
>
> export CC=icc
> export CXX=icpc
> export F77=ifort
> export FC=ifort
> export AR=xiar
> export LD=xild
>
> export CFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> export F77FLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> export FFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> export CXXFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> export MKL="-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread"
>
> VERSION=4.0.1
>
> /scratch/novosirj/install-files/R-${VERSION}/configure --with-blas="$MKL" --with-lapack --prefix=/opt/sw/packages/intel-19_1/R-Project/${VERSION} && \
>        make -j32 && make check && make -j32 install
For what it is worth, we used to build R with the Intel compilers and
MKL on our HPC cluster (on CentOS Linux), and we used the following
setup.  Note the comments about -fp-model precise and -ipo.  It might no
longer be a problem, but maybe worth checking.

fast="-ip -O3 -qopt-mem-layout-trans=3 -xHost -mavx"
## Notes:
## -static and -ipo break compilation
## -no-prec-div breaks make check
## -fp-model precise is needed for make check
## -wd188 removes a lot of warnings (see R Inst. & Adm. manual)
export CC="icc"
export CFLAGS="$fast -wd188 -fp-model precise"
export F77="ifort"
export FFLAGS="$fast -fp-model precise"
export CXX="icpc"
export CXXFLAGS="$fast -fp-model precise"
export FC="ifort"
export FCFLAGS="$fast -fp-model precise"

## This works with intel 2011.10, at least:
BLAS=--with-blas='-mkl=parallel'
LAPACK=--with-lapack

## To make the linker find libraries in modules:
export LDFLAGS=$(echo $LD_LIBRARY_PATH | sed 's/^/-L/; s/:/ -L/g')

./configure "$BLAS" "$LAPACK" --enable-BLAS-shlib --enable-R-shlib
make -j 8
make check
make install

--
Regards,
Bjørn-Helge Mevik

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.2 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7

Ryan Novosielski
> On Jun 25, 2020, at 10:31 AM, Bjørn-Helge Mevik <[hidden email]> wrote:

>
> Signed PGP part
> Ryan Novosielski <[hidden email]> writes:
>
>> Hi there,
>>
>> I initially asked about this on r-help and was told this might be a better venue. I’m not really convinced from reading the posting guide, but I’ll give it a shot. It was also suggested that the R-Project doesn’t really care about building with “non-standard” compilers, but I can’t find any evidence of that on the website (indeed, there’s some mention of successful past builds, and the build itself is successful).
>>
>> I built R 4.0.2 with the Intel Parallel Studio XE compiler suite, versions 19.0.x to 19.1.1. Build seems to go fine. I built it like this:
>>
>> module purge
>> module load intel/19.1.1
>> module list
>>
>> export CC=icc
>> export CXX=icpc
>> export F77=ifort
>> export FC=ifort
>> export AR=xiar
>> export LD=xild
>>
>> export CFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
>> export F77FLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
>> export FFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
>> export CXXFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
>> export MKL="-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread"
>>
>> VERSION=4.0.1
>>
>> /scratch/novosirj/install-files/R-${VERSION}/configure --with-blas="$MKL" --with-lapack --prefix=/opt/sw/packages/intel-19_1/R-Project/${VERSION} && \
>>       make -j32 && make check && make -j32 install
>
> For what it is worth, we used to build R with the Intel compilers and
> MKL on our HPC cluster (on CentOS Linux), and we used the following
> setup.  Note the comments about -fp-model precise and -ipo.  It might no
> longer be a problem, but maybe worth checking.
>
> fast="-ip -O3 -qopt-mem-layout-trans=3 -xHost -mavx"
> ## Notes:
> ## -static and -ipo break compilation
> ## -no-prec-div breaks make check
> ## -fp-model precise is needed for make check
> ## -wd188 removes a lot of warnings (see R Inst. & Adm. manual)
> export CC="icc"
> export CFLAGS="$fast -wd188 -fp-model precise"
> export F77="ifort"
> export FFLAGS="$fast -fp-model precise"
> export CXX="icpc"
> export CXXFLAGS="$fast -fp-model precise"
> export FC="ifort"
> export FCFLAGS="$fast -fp-model precise"
>
> ## This works with intel 2011.10, at least:
> BLAS=--with-blas='-mkl=parallel'
> LAPACK=--with-lapack
>
> ## To make the linker find libraries in modules:
> export LDFLAGS=$(echo $LD_LIBRARY_PATH | sed 's/^/-L/; s/:/ -L/g')
>
> ./configure "$BLAS" "$LAPACK" --enable-BLAS-shlib --enable-R-shlib
> make -j 8
> make check
> make install
Thanks, Bjørn; -ipo no longer breaks compilation (I didn’t specify it, but I see that it was getting added automatically). I was really hoping that "-fp-model precise” (which also implies -prec-div which would seem to affect the other option mentioned) would have an impact on “make check” here, but apparently not. Also -wd188 is no longer required to hide warnings; this doesn’t seem to be happening anymore (but remember it myself from earlier on).

I guess it might make sense to check with Intel; we have support. I wish I could remember if this ever worked right, but I don’t think I thought/knew to run "make check" on our older builds.

--
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
     `'


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.2 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7

Ryan Novosielski
In reply to this post by Ivan Krylov
> On Jun 25, 2020, at 8:06 AM, Ivan Krylov <[hidden email]> wrote:

>
> On Wed, 24 Jun 2020 18:56:06 +0000
> Ryan Novosielski <[hidden email]> wrote:
>
> On my machine, getOption('expressions') is 5000 and the example from
> the test correctly stops with length(traceback()) == 2500. (And the
> simpler example of f <- function() f(); f() stops with
> length(traceback()) == 5000).
>
>> Traceback:
>
> <...>
>
>> 2718: foo()
>
> This (traceback() being more than 2500 entries long) seems to imply
> that the stack size check is somehow skipped. (Perhaps optimized away?)
> The evaluation depth limit is checked in src/main/eval.c, line 705 [*],
> followed by stack size check. Can you attach the debugger and take a
> look at the values of R_EvalDepth and R_Expressions while executing the
> text? What about R_CStackStart and R_CStackLimit? What is the stack
> size limit (ulimit -s?) on the machine running this test?
I can do that. I figure you may have more experience debugging R than I have, but I know the R command is a script, and that running the R binary directly doesn’t result in a working run. So what I’ve tried is to do “gdb /bin/bash” and then running R from that shell. Is that best/are there guidelines? I’m not much of an expert in this area. Is there also any special way I need to compile R to have the appropriate symbols?

The shell I’m running in had 8192 for stack size:

[novosirj@amarel-test1 R-4.0.2-intel-19.1-build]$ ulimit -s
8192

Thanks for your help!

--
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
     `'


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Ryan Novosielski
In reply to this post by Ivan Krylov
> On Jun 25, 2020, at 8:06 AM, Ivan Krylov <[hidden email]> wrote:
>
> On Wed, 24 Jun 2020 18:56:06 +0000
> Ryan Novosielski <[hidden email]> wrote:
>
> On my machine, getOption('expressions') is 5000 and the example from
> the test correctly stops with length(traceback()) == 2500. (And the
> simpler example of f <- function() f(); f() stops with
> length(traceback()) == 5000).
>
>> Traceback:
>
> <...>
>
>> 2718: foo()
>
> This (traceback() being more than 2500 entries long) seems to imply
> that the stack size check is somehow skipped. (Perhaps optimized away?)
> The evaluation depth limit is checked in src/main/eval.c, line 705 [*],
> followed by stack size check. Can you attach the debugger and take a
> look at the values of R_EvalDepth and R_Expressions while executing the
> text? What about R_CStackStart and R_CStackLimit? What is the stack
> size limit (ulimit -s?) on the machine running this test?
>
> --
> Best regards,
> Ivan
>
> [*]
> https://github.com/wch/r-source/blob/8d7ac4699fba640d030703fa010b66bf26054cbd/src/main/eval.c#L705

Thanks again for your help, Ivan, and also Frederick for pointing out how I might run the debugger with R.

I’m finally back at this. Here’s what I see, presuming I’ve done this the right way. Anything of use here as far as troubleshooting?

[novosirj@amarel-test2 bin]$ ./R -d gdb-ia

...

Reading symbols from /scratch/novosirj/install-files/R-4.0.5-intel-19.1-build/bin/exec/R...
(gdb) run
Starting program: /scratch/novosirj/install-files/R-4.0.5-intel-19.1-build/bin/exec/R
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

R version 4.0.5 (2021-03-31) -- "Shake and Throw"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

...

[Detaching after fork from child process 42089]
[Detaching after fork from child process 42091]
> bar <- function() 1+1
> foo <- function() { on.exit(bar()); foo() }
> tryCatch(foo(), error=function(x) TRUE) # now simple "infinite recursion"

Program received signal SIGSEGV, Segmentation fault.
bcEval.R (body=0x3eb7748, rho=0x3f72770, useCache=TRUE) at /scratch/novosirj/install-files/R-4.0.5/src/main/eval.c:6478
6478      codebase = pc = BCCODE(body);
(gdb) print R_EvalDepth
$1 = 2729
(gdb) print R_Expressions
$2 = 5000
(gdb) print R_CStackStart
$3 = 140737488207872
(gdb) print R_CStackLimit
$4 = 7969177
(gdb) quit
A debugging session is active.

        Inferior 1 [process 42083] will be killed.

Quit anyway? (y or n) y

[novosirj@amarel-test2 bin]$ ulimit -s
8192


--
#BlackLivesMatter
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
     `'

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Ivan Krylov
Hello Ryan,

Sorry for not responding right away -- it took me a while to remember
what I meant back then :)

On Fri, 9 Apr 2021 23:34:13 +0000
Ryan Novosielski <[hidden email]> wrote:

> Program received signal SIGSEGV, Segmentation fault.
> bcEval.R (body=0x3eb7748, rho=0x3f72770, useCache=TRUE)
> at /scratch/novosirj/install-files/R-4.0.5/src/main/eval.c:6478
> 6478      codebase = pc = BCCODE(body);
> (gdb) print R_EvalDepth
> $1 = 2729
> (gdb) print R_Expressions
> $2 = 5000
> (gdb) print R_CStackStart
> $3 = 140737488207872
> (gdb) print R_CStackLimit
> $4 = 7969177

> [novosirj@amarel-test2 bin]$ ulimit -s
> 8192

So far I can only say that you get the same ulimit -s of 8192 and
R_CStackLimit of 8192 * 1024 * .95 that I get on my current machine
(and now it's the stack size limit that's reached first, not expression
depth limit). Let's try to get more information.

When you get the SIGSEGV,

1) What does print $_siginfo._sifields._sigfault show? Try printing at
least $_siginfo if the first command gives you an error.

2) When you get the crash, is the body argument accessible? What does
print *body show?

3) What are the addresses of the local variables when the crash
happens? Specifically, what do the following commands show:

print &codebase
print &pc
print R_CStackDir * (R_CStackStart - (uintptr_t)&codebase)
print R_CStackDir * (R_CStackStart - (uintptr_t)&pc)

--
Best regards,
Ivan

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Ryan Novosielski
> On Apr 15, 2021, at 10:35 AM, Ivan Krylov <[hidden email]> wrote:
>
> Hello Ryan,
>
> Sorry for not responding right away -- it took me a while to remember
> what I meant back then :)

There was a long pause there, as I didn’t get to try out those debugging instructions from Frederik right away. They worked great Happy for any assistance whenever I can get it! Thanks!

Was able to get all of that output, though I wish I had a clue what any of it meant. Let me know if there’s other stuff that would be helpful.

I should probably note that I’m running this as: ./R -d gdb-ia from within $BUILDIR/bin. gdb-ia appears to be the Intel copy of gdb (they had their own debugger but eliminated it sometime back).

> So far I can only say that you get the same ulimit -s of 8192 and
> R_CStackLimit of 8192 * 1024 * .95 that I get on my current machine
> (and now it's the stack size limit that's reached first, not expression
> depth limit). Let's try to get more information.
>
> When you get the SIGSEGV,
>
> 1) What does print $_siginfo._sifields._sigfault show? Try printing at
> least $_siginfo if the first command gives you an error.

(gdb) print $_siginfo._sifields._sigfault
$1 = {si_addr = 0x7fffff7fecf8, _addr_lsb = 0, _addr_bnd = {_lower = 0xffff9215f829ff58, _upper = 0x7fffff7fecf8}}

> 2) When you get the crash, is the body argument accessible? What does
> print *body show?

(gdb) print *body
$2 = {sxpinfo = {type = 21, scalar = 0, obj = 0, alt = 0, gp = 0, mark = 0, debug = 0, trace = 0, spare = 0, gcgen = 0, gccls = 0,
    named = 4, extra = 0}, attrib = 0x16fa4f0, gengc_next_node = 0x3eb7710, gengc_prev_node = 0x3eb7780, u = {primsxp = {offset = 44368248},
    symsxp = {pname = 0x2a50178, value = 0x1b66098, internal = 0x16fa4f0}, listsxp = {carval = 0x2a50178, cdrval = 0x1b66098,
      tagval = 0x16fa4f0}, envsxp = {frame = 0x2a50178, enclos = 0x1b66098, hashtab = 0x16fa4f0}, closxp = {formals = 0x2a50178,
      body = 0x1b66098, env = 0x16fa4f0}, promsxp = {value = 0x2a50178, expr = 0x1b66098, env = 0x16fa4f0}}}

> 3) What are the addresses of the local variables when the crash
> happens? Specifically, what do the following commands show:
>
> print &codebase
> print &pc
> print R_CStackDir * (R_CStackStart - (uintptr_t)&codebase)
> print R_CStackDir * (R_CStackStart - (uintptr_t)&pc)

(gdb) print &codebase
$3 = (BCODE **) 0x7fffff7ff360
(gdb) print &pc
$4 = (BCODE **) 0x7fffff7ff358
(gdb) print R_CStackDir * (R_CStackStart - (uintptr_t)&codebase)
$5 = 18446744073701307232
(gdb) print R_CStackDir * (R_CStackStart - (uintptr_t)&pc)
$6 = 18446744073701307224

--
#BlackLivesMatter
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
     `'

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Ivan Krylov
On Thu, 15 Apr 2021 22:46:56 +0000
Ryan Novosielski <[hidden email]> wrote:

> (gdb) print $_siginfo._sifields._sigfault
> $1 = {
>  si_addr = 0x7fffff7fecf8, _addr_lsb = 0,
>  _addr_bnd = {_lower = 0xffff9215f829ff58, _upper = 0x7fffff7fecf8}
> }

> (gdb) print R_CStackDir * (R_CStackStart - (uintptr_t)&codebase)
> $5 = 18446744073701307232

Okay, this is clearly a stack overflow: the faulting address is close
to addresses of other stack variables, and the stack usage, calculated
manually as 140737488207872 - 0x7fffff7ff360, is 8244392, which is
above the (7969177), but the value that gdb gives you looks really
strange. I could only get that value when I calculated
-1 * (140737488207872 - 0x7fffff7ff360) and reinterpreted it as
unsigned.

What is the value of R_CStackDir at the moment of crash? Could it have
somehow became -1 despite the stack growing down?

--
Best regards,
Ivan

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Ryan Novosielski
> On Apr 16, 2021, at 12:12 PM, Ivan Krylov <[hidden email]> wrote:
>
> On Thu, 15 Apr 2021 22:46:56 +0000
> Ryan Novosielski <[hidden email]> wrote:
>
>> (gdb) print $_siginfo._sifields._sigfault
>> $1 = {
>> si_addr = 0x7fffff7fecf8, _addr_lsb = 0,
>> _addr_bnd = {_lower = 0xffff9215f829ff58, _upper = 0x7fffff7fecf8}
>> }
>
>> (gdb) print R_CStackDir * (R_CStackStart - (uintptr_t)&codebase)
>> $5 = 18446744073701307232
>
> Okay, this is clearly a stack overflow: the faulting address is close
> to addresses of other stack variables, and the stack usage, calculated
> manually as 140737488207872 - 0x7fffff7ff360, is 8244392, which is
> above the (7969177), but the value that gdb gives you looks really
> strange. I could only get that value when I calculated
> -1 * (140737488207872 - 0x7fffff7ff360) and reinterpreted it as
> unsigned.
>
> What is the value of R_CStackDir at the moment of crash? Could it have
> somehow became -1 despite the stack growing down?

Well it definitely somehow could have, since it did:

Program received signal SIGSEGV, Segmentation fault.
bcEval.R (body=0x3eb7748, rho=0x3f72770, useCache=TRUE) at /scratch/novosirj/install-files/R-4.0.5/src/main/eval.c:6478
6478      codebase = pc = BCCODE(body);

(gdb) print R_CStackDir
$1 = -1

--
#BlackLivesMatter
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
     `'

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Ivan Krylov
On Fri, 16 Apr 2021 18:06:51 +0000
Ryan Novosielski <[hidden email]> wrote:

> Well it definitely somehow could have, since it did

Wow! This is strange, but at least it should be easy to fix. Try editing
the config.site file in the root of the R source directory and setting
R_C_STACK_DIRECTION=down there. (Make sure to uncomment the line!)
Re-run .../configure, make sure that src/include/config.h contains
#define C_STACK_DIRECTION 1 and build R again.

Does the crash go away? If it does, are you interested in finding out
what went wrong in the configure test for stack direction?

There are lines in m4/R.m4 that I'm not sure I understand:

    if test ${?} = 1; then
      r_cv_cstack_direction=down
    elif test ${?} = 1; then
      r_cv_cstack_direction=up
    fi

How can elif branch have the same condition as the if branch? Shouldn't
the second test had been for $? = 255? On the other hand, if the elif
branch was never taken, how did R_CStackDir become -1?

--
Best regards,
Ivan

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Ryan Novosielski
> On Apr 16, 2021, at 2:32 PM, Ivan Krylov <[hidden email]> wrote:
>
> On Fri, 16 Apr 2021 18:06:51 +0000
> Ryan Novosielski <[hidden email]> wrote:
>
>> Well it definitely somehow could have, since it did
>
> Wow! This is strange, but at least it should be easy to fix. Try editing
> the config.site file in the root of the R source directory and setting
> R_C_STACK_DIRECTION=down there. (Make sure to uncomment the line!)
> Re-run .../configure, make sure that src/include/config.h contains
> #define C_STACK_DIRECTION 1 and build R again.
>
> Does the crash go away?

Will give it a shot, thanks, that’s great!

> If it does, are you interested in finding out
> what went wrong in the configure test for stack direction?

I’m interested in putting in the time to make this build/the tests succeed for anyone else who wants to use R with the Intel Parallel Studio/oneAPI for sure. I’m not actually sure whether it makes much difference, vs. gcc, beyond using the MKL for BLAS/LAPACK, but we do pay for this compiler, and try to use it anytime we might get a slight boost out of it. One of the reasons too is that it can do auto-vectorization and build fat binaries for the various processor instruction sets. I don’t know how much that matters in R either, but if we can get the best performance out of our system-wide install from that, we want to do it.

> There are lines in m4/R.m4 that I'm not sure I understand:
>
>    if test ${?} = 1; then
>      r_cv_cstack_direction=down
>    elif test ${?} = 1; then
>      r_cv_cstack_direction=up
>    fi
>
> How can elif branch have the same condition as the if branch? Shouldn't
> the second test had been for $? = 255? On the other hand, if the elif
> branch was never taken, how did R_CStackDir become -1?

IANAm4P, but that does seem like a pretty good question. I guess there’s probably some mode of m4 I could run against that and see if there’s any indication?

--
#BlackLivesMatter
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
     `'

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, errors in "make check" on CentOS 7.7-7.9

Ryan Novosielski
In reply to this post by Ivan Krylov
> On Apr 16, 2021, at 2:32 PM, Ivan Krylov <[hidden email]> wrote:
>
> On Fri, 16 Apr 2021 18:06:51 +0000
> Ryan Novosielski <[hidden email]> wrote:
>
>> Well it definitely somehow could have, since it did
>
> Wow! This is strange, but at least it should be easy to fix. Try editing
> the config.site file in the root of the R source directory and setting
> R_C_STACK_DIRECTION=down there. (Make sure to uncomment the line!)
> Re-run .../configure, make sure that src/include/config.h contains
> #define C_STACK_DIRECTION 1 and build R again.
>
> Does the crash go away?

So, it does, and no other crashes, though the checks fail in a couple of places, and the final status is error: in tests/reg-tests-1d.R, and also reg-packages.R (those are the only two that have .Rout.fail files).

make[3]: Entering directory `/scratch/novosirj/install-files/R-4.0.5-intel-19.1-build/tests'
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/array-subset.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-tests-1a.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-tests-1b.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-tests-1c.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-tests-1d.R' ...make[3]: *** [reg-tests-1d.Rout] Error 1
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-tests-2.R' ... OK
  comparing 'reg-tests-2.Rout' to '/scratch/novosirj/install-files/R-4.0.5/tests/reg-tests-2.Rout.save' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-examples1.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-examples2.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-packages.R' ...make[3]: *** [reg-packages.Rout] Error 1
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/p-qbeta-strict-tst.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/r-strict-tst.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-IO.R' ... OK
  comparing 'reg-IO.Rout' to '/scratch/novosirj/install-files/R-4.0.5/tests/reg-IO.Rout.save' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-IO2.R' ... OK
  comparing 'reg-IO2.Rout' to '/scratch/novosirj/install-files/R-4.0.5/tests/reg-IO2.Rout.save' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-plot.R' ... OK
  comparing 'reg-plot.pdf' to '/scratch/novosirj/install-files/R-4.0.5/tests/reg-plot.pdf.save' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-S4-examples.R' ... OK
running code in '/scratch/novosirj/install-files/R-4.0.5/tests/reg-BLAS.R' ... OK
make[3]: Leaving directory `/scratch/novosirj/install-files/R-4.0.5-intel-19.1-build/tests'
make[2]: *** [test-Reg] Error 2
make[2]: Leaving directory `/scratch/novosirj/install-files/R-4.0.5-intel-19.1-build/tests'
make[1]: *** [test-all-basics] Error 1
make[1]: Target `check' not remade because of errors.
make[1]: Leaving directory `/scratch/novosirj/install-files/R-4.0.5-intel-19.1-build/tests'
make: *** [check] Error 2

I’m a little new to reading these results, so I’m not sure exactly what I’m looking for. I’ve shared them in the event that what I put in this e-mail is useless:

reg-tests-1d.Rout.fail: https://rutgersconnect-my.sharepoint.com/:u:/g/personal/novosirj_oarc_rutgers_edu/EYK2JHWQ1-9Dvu6gK9lrkRIBkEyA4QqkeH7C4gmbAYyBBQ?e=lfGJL7
reg-packages.Rout.fail: https://rutgersconnect-my.sharepoint.com/:u:/g/personal/novosirj_oarc_rutgers_edu/EazCjI6fRnNKhQASFPeySBUBENVpCqCljFg3-sokBZJnAw?e=8lwywe

Anyhow, there appear to be a number of places, if I have this right:

[novosirj@amarel-test2 tests]$ grep -i -B2 ^error reg-packages.Rout.fail
building package exSexpr ...
Converting Rd files to LaTeX
Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet,  :
  pdflatex is not available
Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet,  :
  pdflatex is not available
Error in running tools::texi2pdf()
--
Hmm ... looks like a package
Creating pdf output from LaTeX ...
Error: R CMD build failed (no tarball) for package exSexpr

These maybe seem like they’re OK, and if I don’t have pdf2latex, they’re expected? Unless I should be looking for something else other than “^Error”. That seemed like the only place something could be going wrong.

For the regression tests, these seem like some of them are actual problems, but maybe someone here knows if some are expected?

[novosirj@amarel-test2 R-4.0.5-intel-19.1-build]$ grep -i -B2 "Asserted error" tests/reg-tests-1d.Rout.fail
> op <- options(device=function(...){}) # non-sense device
> tools::assertError(plot.new(), verbose = TRUE)
Asserted error: no active device and default getOption("device") is invalid
> if(no.grid <- !("grid" %in% loadedNamespaces())) requireNamespace("grid")
> tools::assertError(grid::grid.newpage(), verbose = TRUE)
Asserted error: no active device and default getOption("device") is invalid
--
> aKnots <- c(rep(0, 4), c(0.3, 0.5, 0.6), rep(1, 4))
> tools::assertError(splines::splineDesign(aKnots, x, derivs = 4), verbose = TRUE)
Asserted error: derivs = 4 >= ord = 4, but should be in {0,..,ord-1}
--
> ## Failed to work after r76382--8:
> tools::assertError(formula("3"), verbose=TRUE)
Asserted error: invalid formula "3": not a call
--
> noC <- structure(datasets::trees, class = c("noCol", "data.frame"))
> tools::assertError( noC[1,2], verbose=TRUE) # fails indeed
Asserted error:  [!] Column subscripts not allowed
--
> t2 <- lapply(Alis, head, 2)
> tools::assertError( head(Alis[[1]], c(1, NA)), verbose=TRUE)
Asserted error: invalid 'n' - length(n) must be <= length(dim(x)), got 2 > 1
> tools::assertError( tail(1:5, c(1, NA)), verbose=TRUE)
Asserted error: invalid 'n' - must have length one when dim(x) is NULL, got 2
--
> Foolis <- lapply(Alis, `class<-`, "foo")
> tools::assertError( head(Foolis[[1]], c(1, NA)), verbose=TRUE)
Asserted error: invalid 'n' - length(n) must be <= length(dim(x)), got 2 > 1
--
> tools::assertError(cbind(ts(1:2, start = 0.5, end = 1.5),
+                        ts(1:2, start = 0  , end = 1)), verbose=TRUE)
Asserted error: not all series have the same phase
--
> ## stopifnot() custom message now via <named> args:
> e <- tools::assertError(stopifnot("ehmm, you must be kidding!" = 1 == 0), verbose=TRUE)
Asserted error: ehmm, you must be kidding!
--
> e2 <- tools::assertError(
+  stopifnot("2 is not approximately 2.1" = all.equal(2, 2.1)), verbose=TRUE)
Asserted error: 2 is not approximately 2.1

Thanks again for your help!

--
#BlackLivesMatter
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
     `'

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Ivan Krylov
In reply to this post by Ryan Novosielski
On Fri, 16 Apr 2021 18:39:04 +0000
Ryan Novosielski <[hidden email]> wrote:

> I guess there’s probably some mode of m4 I could run against that and
> see if there’s any indication?

Here's a shell script that should be doing the same thing that
R's .../configure does, but a bit more verbose:

-----------------------------------8<-----------------------------------
#!/bin/sh
cat > conftest1.c <<EOF
#include <stdint.h>
uintptr_t dummy_ii(void)
{
    int ii;

    /* This is intended to return a local address. We could just return
       (uintptr_t) &ii, but doing it indirectly through ii_addr avoids
       a compiler warning (-Wno-return-local-addr would do as well).
    */
    volatile uintptr_t ii_addr = (uintptr_t) &ii;
    return ii_addr;
}
EOF
cat > conftest.c <<EOF
#include <stdio.h>
#include <stdint.h>
extern uintptr_t dummy_ii(void);

typedef uintptr_t (*dptr_type)(void);
volatile dptr_type dummy_ii_ptr;

int main(int ac, char **av)
{
    int i;
    dummy_ii_ptr = dummy_ii;
       
    /* call dummy_ii via a volatile function pointer to prevent
       inlinining in case the tests are accidentally built with
       LTO */
    uintptr_t ii = dummy_ii_ptr();
#ifndef EXACTLY_AS_R_CONFIGURE
    printf(
        "main = %p, sub = %p, main %c sub, ret = %d\n",
        &i, (void*)ii,
        ((uintptr_t)&i > ii) ? '>' : ((uintptr_t)&i < ii) ? '<' : '=',
        ((uintptr_t)&i > ii) ? 1 : -1
    );
#endif
    /* 1 is downwards */
    return ((uintptr_t)&i > ii) ? 1 : -1;
}
EOF
echo "${CC:=cc} ${CPPFLAGS} ${CFLAGS} ${LDFLAGS} ${MAIN_LDFLAGS}" \
    "-o conftest conftest.c conftest1.c"
${CC} ${CPPFLAGS} ${CFLAGS} ${LDFLAGS} ${MAIN_LDFLAGS} -o conftest \
    conftest.c conftest1.c || exit $?
echo ./conftest
./conftest
ret=$?
echo "./conftest exited with $ret"
if test ${ret} = 1; then
  echo r_cv_cstack_direction=down
elif test ${ret} = 1; then
  echo r_cv_cstack_direction=up
fi
exit 0
-----------------------------------8<-----------------------------------

Please run it similarly to the way you run .../configure:

export CC=icc
export CFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
sh .../runme.sh

Does it give the right answer, that is, r_cv_cstack_direction=down?
Does the answer change if you add -DEXACTLY_AS_R_CONFIGURE to CFLAGS?
If the answer is always "down" and you have reused the build directory
(keeping the config.site file between .../configure runs), this is going
to be hard to track down. If you manage to get the "up" answer, it
should be possible to tweak the test until ICC doesn't optimise it to
the point of confusing the stack growth direction.

Either way, I think that the elif branch in the R_STACK_DIRECTION test
should be testing for $? = 255, not 1. I'm almost convinced that the
behaviour would be incorrect on platforms where the test exits with -1.

--
Best regards,
Ivan

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, errors in "make check" on CentOS 7.7-7.9

Ivan Krylov
In reply to this post by Ryan Novosielski
On Sat, 17 Apr 2021 00:13:42 +0000
Ryan Novosielski <[hidden email]> wrote:

> reg-tests-1d.Rout.fail:
> https://rutgersconnect-my.sharepoint.com/:u:/g/personal/novosirj_oarc_rutgers_edu/EYK2JHWQ1-9Dvu6gK9lrkRIBkEyA4QqkeH7C4gmbAYyBBQ?e=lfGJL7
> reg-packages.Rout.fail:
> https://rutgersconnect-my.sharepoint.com/:u:/g/personal/novosirj_oarc_rutgers_edu/EazCjI6fRnNKhQASFPeySBUBENVpCqCljFg3-sokBZJnAw?e=8lwywe

Sorry, these links seem to be asking me to log in. Could you try a
"paste bin" service like https://paste.debian.net/?

> These maybe seem like they’re OK, and if I don’t have pdf2latex,
> they’re expected?

I've never tried to build R without TeX Live installed. Is there
anything about LaTeX-less installation in 'R Installation and
Administration'?

> For the regression tests, these seem like some of them are actual
> problems, but maybe someone here knows if some are expected?

Anything that crashes (well, raises an R error, not crashes the R
process) inside tools::assertError(...) is meant to do that. In fact,
you get an error if it doesn't crash:

tools::assertError(stop('I will crash'))
tools::assertError(stop('I will crash'), verbose = TRUE)
# Asserted error: I will crash
tools::assertError(2+2)
# Error: Failed to get error in evaluating 2 + 2

--
Best regards,
Ivan

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Tomas Kalibera
In reply to this post by Ivan Krylov
Thank you Ryan and Ivan for reporting and debugging this. Godbolt.org
shows that icc 19.0.1 with -O2 -ipo (inter-procedural optimizations) is
too smart and optimizes this stack growth detection code so that it
returns incorrect result on x86_64. Detecting stack growth from C is
tricky - in principle, it cannot be done correctly in a portable way. As
the compilers are getting more sophisticated, it is increasingly more
difficult. We'll have to improve the test, perhaps re-using some or
completely remove it and assume stack grows down. I doubt anyone today
would run a real, non-emulated, system running R where stack would grow up.

(and yes, the elif conditional should use 255)

Tomas


On 4/17/21 11:40 AM, Ivan Krylov wrote:

> On Fri, 16 Apr 2021 18:39:04 +0000
> Ryan Novosielski <[hidden email]> wrote:
>
>> I guess there’s probably some mode of m4 I could run against that and
>> see if there’s any indication?
> Here's a shell script that should be doing the same thing that
> R's .../configure does, but a bit more verbose:
>
> -----------------------------------8<-----------------------------------
> #!/bin/sh
> cat > conftest1.c <<EOF
> #include <stdint.h>
> uintptr_t dummy_ii(void)
> {
>      int ii;
>
>      /* This is intended to return a local address. We could just return
>         (uintptr_t) &ii, but doing it indirectly through ii_addr avoids
>         a compiler warning (-Wno-return-local-addr would do as well).
>      */
>      volatile uintptr_t ii_addr = (uintptr_t) &ii;
>      return ii_addr;
> }
> EOF
> cat > conftest.c <<EOF
> #include <stdio.h>
> #include <stdint.h>
> extern uintptr_t dummy_ii(void);
>
> typedef uintptr_t (*dptr_type)(void);
> volatile dptr_type dummy_ii_ptr;
>
> int main(int ac, char **av)
> {
>      int i;
>      dummy_ii_ptr = dummy_ii;
>          
>      /* call dummy_ii via a volatile function pointer to prevent
>         inlinining in case the tests are accidentally built with
>         LTO */
>      uintptr_t ii = dummy_ii_ptr();
> #ifndef EXACTLY_AS_R_CONFIGURE
>      printf(
>          "main = %p, sub = %p, main %c sub, ret = %d\n",
>          &i, (void*)ii,
>          ((uintptr_t)&i > ii) ? '>' : ((uintptr_t)&i < ii) ? '<' : '=',
>          ((uintptr_t)&i > ii) ? 1 : -1
>      );
> #endif
>      /* 1 is downwards */
>      return ((uintptr_t)&i > ii) ? 1 : -1;
> }
> EOF
> echo "${CC:=cc} ${CPPFLAGS} ${CFLAGS} ${LDFLAGS} ${MAIN_LDFLAGS}" \
>      "-o conftest conftest.c conftest1.c"
> ${CC} ${CPPFLAGS} ${CFLAGS} ${LDFLAGS} ${MAIN_LDFLAGS} -o conftest \
>      conftest.c conftest1.c || exit $?
> echo ./conftest
> ./conftest
> ret=$?
> echo "./conftest exited with $ret"
> if test ${ret} = 1; then
>    echo r_cv_cstack_direction=down
> elif test ${ret} = 1; then
>    echo r_cv_cstack_direction=up
> fi
> exit 0
> -----------------------------------8<-----------------------------------
>
> Please run it similarly to the way you run .../configure:
>
> export CC=icc
> export CFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
> sh .../runme.sh
>
> Does it give the right answer, that is, r_cv_cstack_direction=down?
> Does the answer change if you add -DEXACTLY_AS_R_CONFIGURE to CFLAGS?
> If the answer is always "down" and you have reused the build directory
> (keeping the config.site file between .../configure runs), this is going
> to be hard to track down. If you manage to get the "up" answer, it
> should be possible to tweak the test until ICC doesn't optimise it to
> the point of confusing the stack growth direction.
>
> Either way, I think that the elif branch in the R_STACK_DIRECTION test
> should be testing for $? = 255, not 1. I'm almost convinced that the
> behaviour would be incorrect on platforms where the test exits with -1.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, errors in "make check" on CentOS 7.7-7.9

Ryan Novosielski
In reply to this post by Ivan Krylov
> On Apr 17, 2021, at 5:52 AM, Ivan Krylov <[hidden email]> wrote:
>
> On Sat, 17 Apr 2021 00:13:42 +0000
> Ryan Novosielski <[hidden email]> wrote:
>
>> reg-tests-1d.Rout.fail:
>> https://rutgersconnect-my.sharepoint.com/:u:/g/personal/novosirj_oarc_rutgers_edu/EYK2JHWQ1-9Dvu6gK9lrkRIBkEyA4QqkeH7C4gmbAYyBBQ?e=lfGJL7
>> reg-packages.Rout.fail:
>> https://rutgersconnect-my.sharepoint.com/:u:/g/personal/novosirj_oarc_rutgers_edu/EazCjI6fRnNKhQASFPeySBUBENVpCqCljFg3-sokBZJnAw?e=8lwywe
>
> Sorry, these links seem to be asking me to log in. Could you try a
> "paste bin" service like https://paste.debian.net/?

Sorry about that; I had it set to totally public and tried with a private browser session, but I guess that’s “Awful365” for you. I tried actual Pastebin and it told me that the reg-tests-1d.Rout.fail was offensive. https://paste.debian.net says it’s too large. Let’s give that another whack; both are here: http://www.rnovosielski.ftml.net/r-project/

>> These maybe seem like they’re OK, and if I don’t have pdf2latex,
>> they’re expected?
>
> I've never tried to build R without TeX Live installed. Is there
> anything about LaTeX-less installation in 'R Installation and
> Administration'?

Doesn’t seem to be required unless you want PDF manuals, which I don’t really care about at all; if it were, I’d expect configure to mention that, though yes, I could have read the manual before I got started (by now, who knows, maybe I did; been building R for years).

--
#BlackLivesMatter
____
|| \\UTGERS,   |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - [hidden email]
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
     `'

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, errors in "make check" on CentOS 7.7-7.9

Ivan Krylov
On Sat, 17 Apr 2021 19:21:12 +0000
Ryan Novosielski <[hidden email]> wrote:

> I tried actual Pastebin and it told me that the
> reg-tests-1d.Rout.fail was offensive. https://paste.debian.net says
> it’s too large. Let’s give that another whack; both are here:
> http://www.rnovosielski.ftml.net/r-project/

This link works fine for me, thanks!

For some reason, this build of R fails a test for LAPACK accidentally
ignoring NAs when computing Frobenius norms of matrices:

https://github.com/wch/r-source/commit/db10ee5237b1f9db83a693903c4293650a43244a
https://github.com/wch/r-source/commit/2f546cf778ae3bae8ef2e82c613658c72098a528

Does the following program print NaN on the last line of its output on
your machine?

program testdlange
use ieee_arithmetic, only: ieee_value, ieee_quiet_nan

intrinsic transpose

interface
    double precision function dlange (NORM, M, N, A, LDA, WORK)
        character :: NORM
        integer :: M
        integer :: N
        double precision, dimension( lda, * ) :: A
        integer :: LDA
        double precision, dimension( * ) :: WORK
    end function
end interface

double precision, dimension(2,3) :: A
double precision :: norm

    A = reshape([0, 0, 0, 0, 0, -1], shape(A))
    print '(3f5.1)', transpose(A)

    norm = dlange('F', size(A, 1), size(A, 2), A, size(A, 1), [0D0])
    print *, norm

    A(1,2) = ieee_value(A(1,2), ieee_quiet_nan)
    print '(3f5.1)', transpose(A)

    norm = dlange('F', size(A, 1), size(A, 2), A, size(A, 1), [0D0])
    print *, norm

end program

--
Best regards,
Ivan

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R 4.0.1-4.0.5 built with Intel Composer 19.0-19.1.1, error in "make check" on CentOS 7.7-7.9

Tomas Kalibera
In reply to this post by Tomas Kalibera
On 4/17/21 6:55 PM, Tomas Kalibera wrote:

> Thank you Ryan and Ivan for reporting and debugging this. Godbolt.org
> shows that icc 19.0.1 with -O2 -ipo (inter-procedural optimizations)
> is too smart and optimizes this stack growth detection code so that it
> returns incorrect result on x86_64. Detecting stack growth from C is
> tricky - in principle, it cannot be done correctly in a portable way.
> As the compilers are getting more sophisticated, it is increasingly
> more difficult. We'll have to improve the test, perhaps re-using some
> or completely remove it and assume stack grows down. I doubt anyone
> today would run a real, non-emulated, system running R where stack
> would grow up.
>
> (and yes, the elif conditional should use 255)

I've updated the test for direction of stack growth in R-devel.
If there are any remaining issues, please report.

Thanks
Tomas

>
> Tomas
>
>
> On 4/17/21 11:40 AM, Ivan Krylov wrote:
>> On Fri, 16 Apr 2021 18:39:04 +0000
>> Ryan Novosielski <[hidden email]> wrote:
>>
>>> I guess there’s probably some mode of m4 I could run against that and
>>> see if there’s any indication?
>> Here's a shell script that should be doing the same thing that
>> R's .../configure does, but a bit more verbose:
>>
>> -----------------------------------8<-----------------------------------
>> #!/bin/sh
>> cat > conftest1.c <<EOF
>> #include <stdint.h>
>> uintptr_t dummy_ii(void)
>> {
>>      int ii;
>>
>>      /* This is intended to return a local address. We could just return
>>         (uintptr_t) &ii, but doing it indirectly through ii_addr avoids
>>         a compiler warning (-Wno-return-local-addr would do as well).
>>      */
>>      volatile uintptr_t ii_addr = (uintptr_t) &ii;
>>      return ii_addr;
>> }
>> EOF
>> cat > conftest.c <<EOF
>> #include <stdio.h>
>> #include <stdint.h>
>> extern uintptr_t dummy_ii(void);
>>
>> typedef uintptr_t (*dptr_type)(void);
>> volatile dptr_type dummy_ii_ptr;
>>
>> int main(int ac, char **av)
>> {
>>      int i;
>>      dummy_ii_ptr = dummy_ii;
>>               /* call dummy_ii via a volatile function pointer to
>> prevent
>>         inlinining in case the tests are accidentally built with
>>         LTO */
>>      uintptr_t ii = dummy_ii_ptr();
>> #ifndef EXACTLY_AS_R_CONFIGURE
>>      printf(
>>          "main = %p, sub = %p, main %c sub, ret = %d\n",
>>          &i, (void*)ii,
>>          ((uintptr_t)&i > ii) ? '>' : ((uintptr_t)&i < ii) ? '<' : '=',
>>          ((uintptr_t)&i > ii) ? 1 : -1
>>      );
>> #endif
>>      /* 1 is downwards */
>>      return ((uintptr_t)&i > ii) ? 1 : -1;
>> }
>> EOF
>> echo "${CC:=cc} ${CPPFLAGS} ${CFLAGS} ${LDFLAGS} ${MAIN_LDFLAGS}" \
>>      "-o conftest conftest.c conftest1.c"
>> ${CC} ${CPPFLAGS} ${CFLAGS} ${LDFLAGS} ${MAIN_LDFLAGS} -o conftest \
>>      conftest.c conftest1.c || exit $?
>> echo ./conftest
>> ./conftest
>> ret=$?
>> echo "./conftest exited with $ret"
>> if test ${ret} = 1; then
>>    echo r_cv_cstack_direction=down
>> elif test ${ret} = 1; then
>>    echo r_cv_cstack_direction=up
>> fi
>> exit 0
>> -----------------------------------8<-----------------------------------
>>
>> Please run it similarly to the way you run .../configure:
>>
>> export CC=icc
>> export CFLAGS="-O3 -ipo -qopenmp -axAVX,CORE-AVX2,CORE-AVX512"
>> sh .../runme.sh
>>
>> Does it give the right answer, that is, r_cv_cstack_direction=down?
>> Does the answer change if you add -DEXACTLY_AS_R_CONFIGURE to CFLAGS?
>> If the answer is always "down" and you have reused the build directory
>> (keeping the config.site file between .../configure runs), this is going
>> to be hard to track down. If you manage to get the "up" answer, it
>> should be possible to tweak the test until ICC doesn't optimise it to
>> the point of confusing the stack growth direction.
>>
>> Either way, I think that the elif branch in the R_STACK_DIRECTION test
>> should be testing for $? = 255, not 1. I'm almost convinced that the
>> behaviour would be incorrect on platforms where the test exits with -1.
>>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel