How could I change the "sum" accuracy?

Community Support
1

Hi, I am using KDB+ 3.5, and I have a question about the accuracy of “sum”.

For example, I have a list of float number, and the actual sum value is 1699950.92. But when I use KDB+ to calculate the sum value, it shows 1699951f.

The code shows below:

    test:(145570 621250 229750 191143.2 461635.4 -145570 -325572 149744.5 69962.85 115413.9 -1050000 353084.2 61710 -175000 675667.5 321161.3);

    sum test

How could I solve this problem?

2

Hi Tongwei,

The number is being calculated correctly; increasing the display precision will show the whole number

// Default precision

q)\P

7i

q)sum test

1699951f

q)\P 10

q)sum test

1699950.85

‘-P x’ on the command line will set this at startup, where x is required precision.

Thanks

3

Hi,
Its display problem and not the calculation one. Change your display precision using command  \P. By default it is set to 7.

Ex:

\P  14

sum test  

/ output   1699950.85

More details on wiki:  http://code.kx.com/q/ref/cmdline/#-p-display-precision

4

Thanks for the answers, I do appreciate your time.

? 2017?11?29??? UTC+8??7:12:30?Tongwei Liu???

Hi, I am using KDB+ 3.5, and I have a question about the accuracy of “sum”.

For example, I have a list of float number, and the actual sum value is 1699950.92. But when I use KDB+ to calculate the sum value, it shows 1699951f.

The code shows below:

    test:(145570 621250 229750 191143.2 461635.4 -145570 -325572 149744.5 69962.85 115413.9 -1050000 353084.2 61710 -175000 675667.5 321161.3);

    sum test

How could I solve this problem?