Appending to a file without reading the entire content

Community Support
1

I’m new to KDB ( sorry if this question is dumb). I’m creating the following table

dsPricing:([id:int$(); date:date$()] open:float$();close:float$();high:float$();low:float$();volume:`int$())

q)dsPricing:([id:int$(); date:date$()] open:float$();close:float$();high:float$();low:float$();volume:`int$())

q)`dsPricing insert(123;2003.03.23;1.0;3.0;4.0;2.0;1000)

q)`dsPricing insert(123;2003.03.24;1.0;3.0;4.0;2.0;2000)

q)save `:dsPricing

Let’s say after saving I exit. After starting q, I like to add another pricing item in there without loading the entire file because the file could be large

q)`dsPricing insert(123;2003.03.25;1.0;3.0;4.0;2.0;1500)

I’ve been looking at .Q.dpft but I can’t really figure it out. Also this table/file doesn’t need to be partitioned.


Thanks

2

you can append to a splayed table; it won’t rewrite the column file unless it is a non-mappable type or it has an attribute.

splays can’t have keys (although you can add them very cheaply when reading).

btw, insert is more forgiving than upsert with the int/long mismatch. Hence the literals 123i,2000i.

Although not necessary for your example, I added .Q.en here as you’d need that if you had a column of symbols in future.

q)dsPricing:(id:int$(); date:date$(); open:float$();close:float$();high:float$();low:float$();volume:`int$())

q)`dsPricing insert(123;2003.03.23;1.0;3.0;4.0;2.0;1000)

,0

q)`dsPricing insert(123;2003.03.24;1.0;3.0;4.0;2.0;2000)

,1

q):dsPricing/ set .Q.en[:.;dsPricing]

`:dsPricing/

q)`:dsPricing/ upsert (123i;2003.03.24;1.0;3.0;4.0;2.0;2000i)

`:dsPricing/

q)iddate xkey select from get`:dsPricing/

id  date      | open close high low volume

--------------| --------------------------

123 2003.03.23| 1    3     4    2   1000  

123 2003.03.24| 1    3     4    2   2000  

123 2003.03.24| 1    3     4    2   2000  

3

What about using upsert with path.
Does it load the entire database? I dont think so..

Eg: From http://code.kx.com/wiki/JB:KdbplusForMortals/loading_tables_from_stored_data#1.1.1_Serialized_Tables

.[`:/data/t;();,;] ([] s:`d`e; v:400 500)`:/qdata/t `:/data/t upsert `s`v!(`f;600)`:/data/t get `:/data/ts v-----a 100b 200c 300d 400e 500f 600
4

if the table is not splayed, it will read the table in full, and rewrite.
The example you quote is not splayed.