fix potential integer overflow in mantelhaen.test ?

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

fix potential integer overflow in mantelhaen.test ?

  Someone who was running into an integer overflow warning (and
downstream error) with mantelhaen.test() on a large data set asked about
it on StackOverflow:

  Coercing ntot to double (i.e. ntot -> as.numeric(ntot)) on lines 283
and 285 here:

  seems to fix the problem, fairly harmlessly (*maybe* there are some
edge cases where the integer and floating-point computations give
different answers??? these values are going to get sent to qr.solve() a
few lines later in any case ...)

   If people think this is worthwhile I could submit a bug report.

   Ben Bolker

[hidden email] mailing list