How to traverse all of the text files

Community Support
1

Hi,

suppose:

q)dir1:`:/test;

q)key dir1

2014120120141202`…

q)dir2:` sv dir,(key dir1) 0

q)key dir2

IBM.txtAMD.txt`…

I should how to iterate through all the text files and generate partition hdb?

 

Best regards,

Sky.

2

Here is a very good twiki on loading from a list of csvs and creating as hdb…

http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles#Data\_Loading\_Example

just modify your read to read in *.txt files

hope it goes well,

Sean

3

Hi Sean,
Thanks.
Maybe you misunderstood my meaning.I’m not reading large file.
I want to read many files in different folders.

Best Regards,

Sky

? 2015?4?13??? UTC+8??5:04:24?Sean O’Hagan???

Here is a very good twiki on loading from a list of csvs and creating as hdb…

http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles#Data_Loading_Example

just modify your read to read in *.txt files

hope it goes well,

Sean

4

.util.tree   tree:{ $[x ~ k:key x; x; 11h = type k; raze (. z.s ` sv x,) each k;()]} The tree function accepts a file path as its only parameter and returns all the files found within the directory or any sub-directory.

Psaris, Nick (2015-03-19). Q Tips: Fast, Scalable and Maintainable Kdb+ (Kindle Locations 5861-5866). Vector Sigma. Kindle Edition. 

5

Hi,

I provided that link because it has the ability to load a list of files from an inputted directory. You just need to modify it to accept multiple dirs.

Yes the link says “Loading From Large Files” , but the actual script is just a loader…http://code.kx.com/wsvn/code/cookbook_code/dataloader/loader.q.

The script also generates a partitioned hdb…which is what your question was.

Whilst we are here, lets see what we can do…

//create 2 dirs for files

cd /var/tmp

for j in seano seano2 ; do for i in file1 file2; do touch $j/$i ; done ; done

//pass in dirs into a cmd line param

q -dir /var/tmp/seano /var/tmp/seano2

//create file list from these two dirs

sv/: raze foo,''key each foo:hsym each$.Q.opt[.z.x]`dir

Or focusing on the script…

currently it has 

inputdir:`:examplecsv

loadallfiles[inputdir;1b]

Change it to

inputdir::examplecsv:examplecsv2

loadallfiles[;1b] each inputdir

Thanks,

Sean

6

? 2015?4?15??? UTC+8??9:58:52?Sean O’Hagan???

Hi,

I provided that link because it has the ability to load a list of files from an inputted directory. You just need to modify it to accept multiple dirs.

Yes the link says “Loading From Large Files” , but the actual script is just a loader…http://code.kx.com/wsvn/code/cookbook_code/dataloader/loader.q.

The script also generates a partitioned hdb…which is what your question was.

Whilst we are here, lets see what we can do…

//create 2 dirs for files

cd /var/tmp

for j in seano seano2 ; do for i in file1 file2; do touch $j/$i ; done ; done

//pass in dirs into a cmd line param

q -dir /var/tmp/seano /var/tmp/seano2

//create file list from these two dirs

sv/: raze foo,''key each foo:hsym each$.Q.opt[.z.x]`dir

Or focusing on the script…

currently it has 

inputdir:`:examplecsv

loadallfiles[inputdir;1b]

Change it to

inputdir::examplecsv:examplecsv2

loadallfiles[;1b] each inputdir

Thanks,

Sean

On Monday, April 13, 2015 at 9:16:58 AM UTC+1, huangyu...@gmail.com wrote:

Hi,

suppose:

q)dir1:`:/test;

q)key dir1

2014120120141202`…

q)dir2:` sv dir,(key dir1) 0

q)key dir2

IBM.txtAMD.txt`…

I should how to iterate through all the text files and generate partition hdb?

 

Best regards,

Sky.

Hi sean,Thank you very much.