How to perform the following functions batch

Community Support
1

Hi All,


I wrote a function to read the text and create the table, but I do not know how to batch execution.Can you help me?


d:`:/home/data

types:“SZSEEEHIBII”

name:$ "\t" vs raze read0:/home/columns.txt

f:{[x;y];flip name!(types;“\t”)0: sv ( sv d,(key d) x),(key (` sv d,(key d) x)) y};


Note: There are a lot of text files in the file path


Thank you


Sky

2

You should take a look at .Q.fs (http://code.kx.com/wiki/DotQ/DotQDotfs) as well as the Cookbook article (http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles).

3

? 2015?2?11??? UTC+8??7:33:24?Flying???

You should take a look at .Q.fs (http://code.kx.com/wiki/DotQ/DotQDotfs) as well as the Cookbook article (http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles).

4

I think I misunderstood your question. I thought you were thinking that columns.txt was too large to be loaded in one go.

If your columns.txt provides only a list of filenames, it’d be relatively straightforward to load each file in sequence. Just keep enumerating and upserting your table while going through your file list.

5

Hi,

If you are looking for a way to enumerate all of the files in the directory, you can just
use the key function to get a list of all the files and use .Q.dd to concatenate the directory
path to the front.

    .Q.dd[dir;] each key dir

You can then easily run your function over that list with something like

    f:{flip name!(types;“\t”)0:x}
    raze f each .Q.dd[d;] each key

Thanks

Mark Rooney
Financial Software Developer
AQUAQ Analytics
On Wednesday, 11 February 2015 08:24:59 UTC, huangyu...@gmail.com wrote:

Hi All,


I wrote a function to read the text and create the table, but I do not know how to batch execution.Can you help me?


d:`:/home/data

types:“SZSEEEHIBII”

name:$ "\t" vs raze read0:/home/columns.txt

f:{[x;y];flip name!(types;“\t”)0: sv ( sv d,(key d) x),(key (` sv d,(key d) x)) y};


Note: There are a lot of text files in the file path


Thank you


Sky

6

Sorry, that last line should be:

    raze f each .Q.dd[d;] each key d

7

Hi Flying,Thanks very much! 

? 2015?2?12??? UTC+8??10:21:15?Flying???

I think I misunderstood your question. I thought you were thinking that columns.txt was too large to be loaded in one go.

If your columns.txt provides only a list of filenames, it’d be relatively straightforward to load each file in sequence. Just keep enumerating and upserting your table while going through your file list.

8

Hi Mark,

Your help  is very useful?thank you very much.

But,in this path there are other paths, as follows:

:/home/data/20050201

:/home/data/20050202

…

:/home/data/yyyymmdd

q)key d

2005020120050202…


q)key (` sv d,(key d) 1)

AAPL_20050201.txtIBM_20050201.txt…

Best regards,

Sky

? 2015?2?12??? UTC+8??5:55:22?mark…@aquaq.co.uk???

Sorry, that last line should be:

    raze f each .Q.dd[d;] each key d