Re: gcc strangeness

• Previous message: Ted Unangst: "out of bounds in local_apic.c"
• In reply to: Dmitry Morozovsky: "gcc strangeness"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Date: Wed, 14 Jul 2004 10:54:44 +1000
To: Dmitry Morozovsky <marck@rinet.ru>

On 2004-Jul-11 21:06:32 +0400, Dmitry Morozovsky <marck@rinet.ru> wrote:
>marck@woozle:/tmp/tsostik> cat x.c
>#include <stdio.h>
>int main ()
>{
> float a;
> for(a=0.01;a<=0.1; a+=0.01)
> printf("%f %.3f %d\n", a*100, a*100, (int)(a*100));
>return 0;
>}
>marck@woozle:/tmp/tsostik> cc x.c
>marck@woozle:/tmp/tsostik> ./a.out
>1.000000 1.000 0
>2.000000 2.000 1
>3.000000 3.000 2
>4.000000 4.000 3
>5.000000 5.000 5
>6.000000 6.000 6
>7.000000 7.000 7
>8.000000 8.000 7
>9.000000 9.000 8
>9.999999 10.000 9

As various other people have pointed out, this is an artifact of
binary floating point arithmetic, rather than gcc. Note that
single-precision floating point only gives you 24 binary (or 7.22
decimal) digits of precision. Expecting to accurately display 7
digits may be somewhat hopeful.

The correct solution is to use rint(3), rather than casting to int
(which is truncation). ceil(3) will solve this specific issue but
is just as susceptable to unexpected rounding.

Switching to double precision will help but is still not a complete
solution. I looked into a similar problem this a few years ago
following a complaint that int(0.29 * 100) == 28 in double precision.
The following is my analysis of that problem.

The underlying problem is that 1/100 is not representable in a finite
number of bits. Specifically,
1/100 = 0d0.01 = 0b0.0000[00101000111101011100] = 0x0.0[28F5C]
(where the brackets enclose the repeating digits).

As an example, 0.29 = 0x0.4[A3D70]. Expressed as a rounded IEEE 754
double precision number, this is 0x3fd28f5c28f5c28f, which
underestimates the correct value by ~2e-17 (0.36ULP[1]). When this
number is multiplied by 100, the underestimate becomes 0.56ULP,
causing the result to round down (ie the result is 1ULP = ~3.4e-15
low). Since conversion-to-integer is defined to round towards zero,
this gives an integer value of 28.

A good (though not hyper-linked) bibliography on floating point can be
found at http://www.cse.nd.edu/~milind/references/floatingpoint. Prof
William Kahan's writings (http://www.cs.berkeley.edu/~wkahan/) also make
interesting reading.

[1] Units in the Last Place.

Peter
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

• Previous message: Ted Unangst: "out of bounds in local_apic.c"
• In reply to: Dmitry Morozovsky: "gcc strangeness"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]