Loading large data into a splay table

Community Support
1

I’m trying to understand what’s going on in the code:

.Q.fs[{ .[`:dir/trade/; (); ,; .Q.en[`:dir] flip colnames!("DFFFFIS";",")0:x]}]`:file.csv

from:

http://code.kx.com/wiki/Cookbook/LoadingFromLargeFilesAndSplaying

  1.  My understanding is that the following reads the chunk of rows passes by .Q.fs, creates a table, and creates (or amends) the sym file bases on the syms in the table and returns the table.

    .Q.en[`:dir] flip colnames!(“DFFFFIS”;“,”)0:x

is the table that is output from this the same as the table output from flip colnames!(“DFFFFIS”;“,”)0:x?  i.e., what’s happening behind the scenes is saving the sym file and then returning the argument?

  1.  In a functional amend .[L;I;f;y], from another thread it appears that I=() means use all of L in function f (i.e., call f(L,y)).  If this is the case, why do you need a functional amend – why not just use a function directly?

  2. Does “,” function applied to two arguments (in this case, `:dir/trade/ and .Q.en[dir] table) mean join the tables and save them down?

Sorry if these are basic questions.  Still learning the language even after a few months of using it!  I tried going through the documentation, but I still had some trouble.

2

Hi Namit,

  1. .Q.en[`:dir] flip colnames!(“DFFFFIS”;“,”)0:x returns the table with the symbol type columns enumerated.  This link (http://code.kx.com/wiki/Cookbook/SplayedTables#Enumeratingvarcharcolumnsinatable) explains what is going on.

2 and 3) The functional amend in this case is basically just upsert :

q)upsert

.[;();,;]

The upsert is taking the result of  .Q.en[:dir] flip colnames!("DFFFFIS";",")0:x</font> and appending it to the table on disk which is located at <font face='"monospace,' monospace>:dir/trade/

Regards,

Paul