# Re: [Rd] Floating point precision / guard digits? (PR#13771)

6 messages
Open this post in threaded view
|
Report Content as Inappropriate

## Re: [Rd] Floating point precision / guard digits? (PR#13771)

 (I am replacing R-devel and r-bugs with r-help as addressees.) On Sat, Jun 20, 2009 at 9:45 AM, Dr. D. P. Kreil <[hidden email]> wrote: > So if I request a calculation of "0.3-0.1-0.1-0.1" and I do not get 0, > that is not an issue of rounding / underflow (or whatever the correct > technical term would be for that behaviour)? > No.  Let's start from the beginning. In binary floating point arithmetic, all numbers are represented as a*2^b, where a and b have a fixed number of digits, so input conversion from decimal form to binary form inherently loses some precision -- that is, it rounds to the nearest binary fraction. For example, representation(0.3) is 5404319552844595 * 2^-54, about 1e-17 less than exactly 3/10, which is of course not representable in the form a*2^b. The EXACT difference (calculating with rationals -- no roundoff errors etc.) between representation(0.3) and 3*representation(0.1) is 2^-55 (about 1e-17); the EXACT difference between representation(0.3) and representation(3*representation(0.1)) is 2^-54.  As it happens, in this case, there is no rounding error at all -- the floating-point result of 0.3 - 3*0.1 is exactly -2^-54.  I thought that guard digits would mean that 0.3-0.1*3 should be calculated > in higher precision than the final representation of the result, i.e., > avoiding that this is not equal to 0? Guard digits and sticky bits are techniques for more accurate rounding of individual arithmetic operations, and do not persist beyond each individual operation.  They cannot create precise results out of imprecise inputs (except when they get lucky!).  And even with precise inputs, they cannot create correctly rounded results with multiple operations.  Consider for example (1.0 + 1.0e-15) - 1.0.  The correctly rounded result of (1.0+1.0e-15) is 1.0000000000000011...  And the correctly rounded result of (1.0+1.0e-15)-1.0 is 1.11e-15, which is 11% different than the mathematical result. Perhaps you are thinking about the case where intermediate results are accumulated in higher-than-normal precision.  This technique only applies in very specialized circumstances, and it not available to user code in most programming languages (including R).  I don't know whether R's sum function uses this technique or some other (e.g. Kahan summation), but it does manage to give higher precision than summation with individual arithmetic operators:     sum(c(2^63,1,-2^63)) => 1 but    Reduce(`+`,c(2^63,1,-2^63)) => 0 I am sorry if I am not from the field... If you can suggest an online > resource to help me use the right vocabulary and better understand the > fundamental concepts, I am of course grateful. I would suggest "What every computer scientist should know about floating-point arithmetic" *ACM Computing Surveys* *23*:1 (March 1991) for the basics.  Anything by Kahan (http://www.cs.berkeley.edu/~wkahan/) is interesting.  Beyond elementary floating-point arithmetic, there is of course the vast field of numerical analysis, which underlies many of the algorithms used by R and other statistical systems.             -s         [[alternative HTML version deleted]] ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: [Rd] Floating point precision / guard digits? (PR#13771)

 Dear Stavros, Thank you very much for your helpful email and your patience. > Perhaps you are thinking about the case where intermediate results are > accumulated in higher-than-normal precision.  This technique only applies in > very specialized circumstances, and it not available to user code in most > programming languages (including R). Ah, that's probably where I went wrong. I thought R would take the "0.1", the "0.3", the "3", convert them to extended precision binary representations, do its calculations, an the reduction to normal double precision binary floats would only happen when the result was stored or printed. Having read your explanations now, I suspect it was unreasonable to expect that. >  I don't know whether R's sum function > uses this technique or some other (e.g. Kahan summation), but it does manage > to give higher precision than summation with individual arithmetic > operators: > >     sum(c(2^63,1,-2^63)) => 1 > but >    Reduce(`+`,c(2^63,1,-2^63)) => 0 That is very interesting! > I would suggest "What every computer scientist should know about > floating-point arithmetic" ACM Computing Surveys 23:1 (March 1991) for the > basics.  Anything by Kahan (http://www.cs.berkeley.edu/~wkahan/) is > interesting.  Beyond elementary floating-point arithmetic, there is of > course the vast field of numerical analysis, which underlies many of the > algorithms used by R and other statistical systems. Thank you very much for the pointers! Best regards, David. ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: [Rd] Floating point precision / guard digits? (PR#13771)

 On Sat, Jun 20, 2009 at 4:10 PM, Dr. D. P. Kreil <[hidden email]> wrote: > Ah, that's probably where I went wrong. I thought R would take the > "0.1", the "0.3", the "3", convert them to extended precision binary > representations, do its calculations, an the reduction to normal > double precision binary floats would only happen when the result was > stored or printed. > This proposal is problematic in many ways.  For example, it would *still* not guarantee that 0.3 - 3*0.1 == 0, since extended-precision floats have the same characteristics as normal-precision floats.  Would you round to normal precision when passing arguments?  Then sqrt could not produce extended-precision results. etc. etc. I suppose R could support an extended-precision floating-point type, but that would require that the *user* choose which operations were in extended-precision and which in normal precision. (And of course it would be a lot of work to add in a complete and consistent way.)            -s         [[alternative HTML version deleted]] ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: [Rd] Floating point precision / guard digits? (PR#13771)

 Yes, I see the problem now! Thank you for bearing with me and for the helpful explanations and info. Best regards, David. 2009/6/20 Stavros Macrakis <[hidden email]>: > On Sat, Jun 20, 2009 at 4:10 PM, Dr. D. P. Kreil <[hidden email]> wrote: >> >> Ah, that's probably where I went wrong. I thought R would take the >> "0.1", the "0.3", the "3", convert them to extended precision binary >> representations, do its calculations, an the reduction to normal >> double precision binary floats would only happen when the result was >> stored or printed. > > This proposal is problematic in many ways.  For example, it would *still* > not guarantee that 0.3 - 3*0.1 == 0, since extended-precision floats have > the same characteristics as normal-precision floats.  Would you round to > normal precision when passing arguments?  Then sqrt could not produce > extended-precision results. etc. etc. > > I suppose R could support an extended-precision floating-point type, but > that would require that the *user* choose which operations were in > extended-precision and which in normal precision. (And of course it would be > a lot of work to add in a complete and consistent way.) > >            -s > ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: [Rd] Floating point precision / guard digits? (PR#13771)

 In reply to this post by macrakis Stavros Macrakis wrote [...] > programming languages (including R).  I don't know whether R's sum function > uses this technique or some other (e.g. Kahan summation), but it does manage > to give higher precision than summation with individual arithmetic > operators: > >     sum(c(2^63,1,-2^63)) => 1 >   not if the arguments are passed as separate elements in ...: x = c(2^63, 1, -2^63) sum(x) # 1 do.call(sum, as.list(x)) # 0 y = 1:3 sum(y) # 6 do.call(sum, as.list(y)) # 6 vQ ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.