User-Agent: Workspace Webmail 5.14.0Message-Id: <20150513073755.85f80dae80d1d2f2e266ec6278e6cbe8.99bb5f948c.wbe@email07.europe.secureserver.net>From: “David Demner (AquaQ)” <david.demner>To: personal-kdbplus@googlegroups.comSubject: RE: [personal kdb+] Sorting without combiningDate: Wed, 13 May 2015 07:37:55 -0700Mime-Version: 1.0
Hi,
Late to the question here but I feel there are a few points worth mentioning…
//define a more realistic table as well as original table
t:(a:11 12t;bid:(1 4 3 2;1 1 3 2);bid_size:(100 100 200 300;100 200 300 100); bid_book:2#enlist CBOE
NYSEARCA
AMEX);
n:10000000;
t2:(a:n?minute$60*0+til 20;bid:n?100;bid_size:n?1000;bid_book:n?
CBOENYSE
ARCA`AMEX);
t3:`a xgroup t2;
//some metrics on first query
//im not a fan of nested/cumulative updates - they can get very long winded
//it obviously still works, just not a fan
q)\ts:1000 delete inds from update bid:bid@'inds, bid_size:bid_size@'inds, bid_book:bid_book@'inds from update inds:iasc each bid from t
5 3712
q)\ts delete inds from update bid:bid@'inds, bid_size:bid_size@'inds, bid_book:bid_book@'inds from update inds:iasc each bid from t3
1422 335547232
//if table is not grouped originally, then you can do a 2 column sort on it prior to grouping. xasc can be very useful for you here.
q)\ts:1000 a xgroup
a`bid xasc ungroup t
23 6480
q)\ts a xgroup
a`bid xasc ungroup t3
3214 1073743312
//alternative - update on top of an exec
q)\ts:1000 ![t;();0b;exec `bid`bid_size`bid_book!enlist each (bid;bid_size;bid_book)@':iasc each bid from t]
4 2688
q)\ts ![t3;();0b;exec `bid`bid_size`bid_book!enlist each (bid;bid_size;bid_book)@':iasc each bid from t3]
921 335546608
//you can also consider the use of fby - just for talk sake
q)\ts ![t3;();0b;exec enlist each `bid`bid_size`bid_book!(bid;bid_size;bid_book)@‘:(iasc’;bid) fby a from t3]
920 335546720
//but if we step back from the problem - it looks like “a” can be your key - if so we can now use the fact that you now make a dict to dict relationship and modify contents of each dict
//this will keep the speed and mem usage down to a minimal
q)\ts:1000 {f,'{flip `bid xasc flip y x}[;x] each f:key x}1!t
26 3008
q)\ts {f,'{flip `bid xasc flip y x}[;x] each f:key x}t3
486 255856784
//confirm match
q)(ungroup {f,'{flip `bid xasc flip y x}[;x] each f:key x}t3)~ungroup delete inds from update bid:bid@'inds, bid_size:bid_size@'inds, bid_book:bid_book@'inds from update inds:iasc each bid from t3
1b
HTH,
Sean