Received: by 10.100.45.5 with SMTP id s5mr599989ans.10.1233983229998; Fri, 06 Feb 2009 21:07:09 -0800 (PST)Date: Fri, 6 Feb 2009 21:07:09 -0800 (PST)X-IP: 119.122.241.10User-Agent: G2/1.0X-Google-Token: 8ZEDiAwAAAANlbU470GZnuiHuoxQzDDbX-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506; .NET CLR 3.5.21022; CIBA; .NET CLR 1.1.4322),gzip(gfe),gzip(gfe)Message-ID: Subject: How to update/delete certain rows from a splayed table on disk?From: “Richard Z.” To: “Kdb+ Personal Developers” X-Google-Approved: charlie@kx.com via web at 2009-02-07 08:54:14For example,1) I builded a test table and saves(splays) it to disk:/start q.exeq) n:10000000; /increase n to increase the table sizeq) t:(id:til n; c1:n?1000j; c2:n?1000j; c3:n?1000j; c4:n?1000j); /generate the test table tq) :d:/qdata/t/ set t; /splays to directory d:/qdata/q , directorycan be changedq) \\ / exit q2)update certain rows:q d:/qdata/t /start q ,load directory d:/qdata/tq) update c1:0j, c2:0j, c3:0j, c4:0j from t where id<5 /updatesome rowsq) `:d:/qdata/t/ set t /saves to diskIt seems that the q process will load all the data of t into memorybefore updating (I found the memory used by q process increased byabout table t’s size) .My question is: how to update/delete the splayed table on disk withoutloading all rows into memory?Any ideas? Thanks in advance.Richard Z.
If you want to update just one column or one column basedon others, you can map the columns you’d like to modify in with get, change it/them and then flush back to disk set. When deleting data you have to read/modify/write every column to keep the vectors the same size so you may as well let q load it in for you. If for some reason memory is an issue (i.e. not partitioned), you can follow the same approach as the update, but in a loop for all columns. A huge splayed table that is not part’ed in some way is probably not recommended though.
Thanks!On Feb 8, 12:51?am, Tim Rieder <trie…> wrote:> If you want to *update *just one column or one column based on others, you> can map the columns you’d like to modify in with get, change it/them and> then flush back to disk set. When *deleting *data you have to> read/modify/write every column to keep the vectors the same size so you may> as well let q load it in for you. If for some reason memory is an issue> (i.e. not partitioned), you can follow the same approach as the update, but> in a loop for all columns. A huge splayed table that is not part’ed in some> way is probably not recommended though.>>>> On Sat, Feb 7, 2009 at 12:07 AM, Richard Z. <drzwz…> wrote:>> > For example,> > 1) I builded a test table and saves(splays) it to disk:>> > /start q.exe> > q) ?n:10000000; ? /increase n to increase the table size> > q) ?t:(id:til n; c1:n?1000j; c2:n?1000j; c3:n?1000j; c4:n?1000j); ?/> > generate the test table t> > q) ?:d:/qdata/t/ set t; ?/splays to directory d:/qdata/q , directory> > can be changed> > q) \\ ? ? / exit q>> > 2)update certain rows:>> > q d:/qdata/t ? ? ? ? /start q ,load directory d:/qdata/t> > q) update c1:0j, c2:0j, c3:0j, c4:0j from t where id<5 ? ? ? ?/update> > some rows> > q) `:d:/qdata/t/ set t ? ? ? ? ?/saves to disk>> > It seems that the q process will load all the data of t into memory> > before updating (I found the memory used by q process increased by> > about table t’s size) .>> > My question is: how to update/delete the splayed table on disk without> > loading all rows into memory?>> > Any ideas? ? Thanks in advance.>> > Richard Z.- Hide quoted text ->> - Show quoted text -</drzwz…></trie…>