comparison tolerance for reals

Community Support
1

The cookbook wiki says that floating point comparison tolerance is
hardwired at 2^-43 times the magnitude of the larger of the two
numbers being compared. That makes sense for 8-byte floats (with 53-
bit mantissas) but is beyond the resolution of 4-byte floats.

Experimentation suggests that there isn’t any tolerance at all when
comparing reals:

q)1.0000002e=1.0000001e
0b
q){reverse -4#-8!x} each 1.0000002 1.0000001e
0x3f800002
0x3f800001

Is there any tolerance for reals?

2

note that you never compute with reals

q)1e+2e

3f

so = compares as floats, hence they are distinct

advice: avoid reals, they are trouble
? Attila

3

On Jan 8, 2:53?am, Attila Vrabecz <attila.vrab…> wrote:> note that you never compute with realsGood point. I suppose reals are there only for storage purposesanyway. I thought there might be a more explicit description of thebehavior from kx, but maybe not.</attila.vrab…>