kdb+ intro question

Community Support
1

Well,

I am currently trying to find the fastest way to access market data from memory for different stock_id for various time period. I am not only open to KDB+ but I have favorised it since it looks to be the standard in the industry.

I have loaded my data from csv file which contains these columns:

Cols: (“ITFIIFFC”;“,”) 0:`MyDataPartial.csv

table: flip stock_idtimelastlast_sizetotal_volumebidaskbasis_for_last!Cols

stock_id time         last    last_size total_volume bid   ask   basis_for_last

2        13:05:20.210 44.07   100       5278320      44.06 44.08 C

2        13:05:20.213 44.07   200       5278520      44.06 44.08 C

2        13:05:20.213 44.07   100       5278620      44.06 44.08 C

2        13:05:22.063 44.0798 600       5279220      44.06 44.08 C

Basically, I have to retreive data for specific time range like this:

 select from table where not time<09:30:00, time < 09:30:05, stock_id in (3,5,7)

I wonder if I can reach better seek time. I currently have an average of 300ms (\t on the beggining of the query).

I do not know for the moment which are the optimisation I could do to achieve better result.

Thank you very much for any recommandations and for reading my post!

Mathieu

2

try variations along the lines of


update s#time,g#stock_id from `table; / if time is sorted

update s#time,p#stock_id from `table; / and if stock_id can be parted, i.e. ids are in contiguous blocks

select from table where time within 09:30:00.000 09:30:04.999,stock_id in 3 5 7

or

select from table where stock_id in 3 5 7,time within 09:30:00.000 09:30:04.999

in general use the most restrictive constraint first in the where clause

3

thanks!