Floating point precision error

Community Support
1

Hi all,

I have encountered floating point precision error.( e.g. 0.1-0.1 !=0, or 0.2 - 0.1 != 0.1)

Understand floating point error is a known issue for floats in many languages.

May I know is there any ways to work around it like the ‘decimal’ data type in some other languages?

Thanks very much.

Cheers,

Juno

2

Hi Juno,

Unfortunately there isn’t an easy work around for this kind of precision error in kdb+.

The finite precision of floats means it’s best to avoid using them for boolean comparision wherever possible.

When you make a boolean comparision of non-zero floats in kdb+ it checks if the values are tolerantly equal, which can compensate for this loss of precision when it is small (but not infallibly).

Comparisons to zero work a little differently, as zero is only tolerantly equal to zero. You can use this to check for precision errors when working with floats.

You can use \P to set the precision displayed to the console for floats, which can help you to see the point at which you’re going to lose precision, and hence means you’re more able to work around these errors.

You can read more on precision here:

http://code.kx.com/q/cookbook/precision/

Hope this helps,

Joseph

3

Hi Joseph,

It’s alright, no problem. I would prefer to change my arithmetic operation from floating point to integer.

And to return the the corresponding value divided by a multiplier on demand.

This avoid checking precision every time it has to do calculations.

Cheers,

Juno