[DGD] Float precision loss

[John West McKenna]

Petter Nyström writes:

>I have run into a behaviour that looks very much like a bug to me.

><> eval 4.3 + 5.0 - 4.3 - 5.0
>$6 = 5.8207661e-11

It's not a bug at all - the fault is in your expectations of the behaviour
of floating point numbers.  I just tried the same thing in Lua and Haskell,
and got a similar result.

4.3 has an exact representation in decimal, but not in binary.  When you
add 5.0, the result is more than double 4.3.  That means it will be
normalised - the mantissa will be shifted and the exponent increased.  
That means bits will be lost.

The problem is similar to this in decimal:

(7.001 + 7.000) - 7.001) - 7.000
= (7.001*10^0 + 7.000*10^0) - 7.001*10^0) - 7.000*10^0
= (1.400*10^1 - 7.001*10^0) - 7.000*10^0
= 6.999*10^0 - 7.000*10^0
= -1.000*10^-3

Compulsory reading for anyone dealing with floating point:
http://citeseer.ist.psu.edu/goldberg91what.html

John