Optimize Kdb group queries ('by' clause) for a speedy execution

Hello,

I need to optimize Kdb group queries to reduce their total execution time. I am executing a group query using ‘by’ clause over 4 columns. The group query itself runs over a subquery. 

The subquery returns around 6 million records and it runs in approximately 700 ms. The group query that runs on top of it reduces these records to around 3-4k, but it takes almost 4000 ms (4 seconds) to run which is beyond the acceptable limit of the application. I need to optimize it so that the total query execution time is less than 1.5 seconds. Is there any performant alternative to ‘by’ clause? Is there something I can do with the grouped columns that will improve the execution time?

Have a reproducible example? It’s probably related to the types you’re grouping on. The below example reduces to 12K rows but performs faster:

q)n:6000000;t:(sym:n?1;date:.z.d+n#1 2 3;mkt:n?1;typ:n?`1;price:n?2.0)

q)count select by date,sym,mkt,typ from t

12288

q)\t select by date,sym,mkt,typ from t

261