Mapping KDB+ database

Community Support
1

Hi guys, I have been struggling creating a function to map an entire kdb+ database. I am staring with a directory and get the type for each files - >type each . Main If type = 98 (table) then save it. If type = 99 (dictionary), open the dictionary and redo the whole thing until I have all my tables mapped. Have you guys an idea about how to go about it? Many thanks in advance.

2

This should work

mapper:{[dir] recur:{$[11h=type d:key x;raze x,.z.s each sv/:x,/:d;d]}; content:recur dir; content where content~'key each content };mapper[:.]

3

Thank you so much, that’s helps me a lot. I am trying to understand what you did here because there is still a type and symbol error. So you key all content and check if it’s type 11 throughout all directories? Could you please explain the sv/:x,/:d;d? Thanks again

4

Thank you so much, that’s helps me a lot.

I am trying to understand what you did here because there is still a type and symbol error. So you key all content and check if it’s type 11 throughout all directories?

  • If you key the path of a file you will get a file back, so a single symbol, if it’s a directory you get a list of the contents, so it’s of type -11h. The logic is for each item returned that is a directory, reapply the function
  • When you say type and symbol errror, do you mean that you are getting a signal for a type error   ’ 'type  ? 
  • How are you executing this function?

Could you please explain the sv/:x,/:d;d?

  • d - the contents of the directory, x - the directory
  • ` sv/:x,/:d - will join each item name in the directory to the directory, which is then converted into a hysm of the file path via an overload of sv 
  • The last item ;d  is just the third parameter for the conditional operator at the start of the function, that’s matching up with the original conditional. In this case, if the item isn’t a directory, output it. 

There is also the option to execute using OS function, although performing this in q is a great exercise for those learning kdb.

  • Windows:
    q)\dir /a-d /b /s
  • Linux
    q)\find .

I hope this clarifies things a bit, feel free to ask any other questions

  • CB
5

Thanks again for taking the time that clarifies things a lot. Yes I am getting a 'type error. I am executing the function with mapper [.main] If I replace 11h by 99h I get a key error

6

Hi Victor

File handles need to be prepended with a colon, so if you wanted to map a directory called .main , you should call this like

mapper[:.main]`

Please give this a shot and let me know what the results are.

  • CB
7

Thanks so I have tried this. I don’t have an error anymore but it gives me no result at all.

8

Try starting a q session in that database 

mapper[:.]`