fix potential integer overflow in mantelhaen.test ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

fix potential integer overflow in mantelhaen.test ?

bbolker
  Someone who was running into an integer overflow warning (and
downstream error) with mantelhaen.test() on a large data set asked about
it on StackOverflow:

https://stackoverflow.com/questions/48422398/na-nan-inf-in-foreign-function-error-with-mantelhaen-test/48428596#48428596

  Coercing ntot to double (i.e. ntot -> as.numeric(ntot)) on lines 283
and 285 here:

https://github.com/wch/r-source/blob/af7f52f70101960861e5d995d3a4bec010bc89e6/src/library/stats/R/mantelhaen.test.R#L283

  seems to fix the problem, fairly harmlessly (*maybe* there are some
edge cases where the integer and floating-point computations give
different answers??? these values are going to get sent to qr.solve() a
few lines later in any case ...)

   If people think this is worthwhile I could submit a bug report.

 cheers
   Ben Bolker

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel