I'm investigating the serialization format for various KDB datatypes as part of some work to maintain an existing library for interfacing with a KDB system.
This library implements a KDB compatible serialization, such that data can be read from and written to disk by both KDB and the language runtime which uses this library.
I have written the string (list of char) "ab" to a file.
When I read the data back using a hex editor, I find a very unusual serialization format.
The first two bytes of data are different to what I would typically expect. `FE, 20` instead of `FF 01`. Most of the other types of data which I have serialized begin with a header containing `FF 01`.
Following this there is a byte `0A` which I believe corresponds to "list of char".
Then there are a sequence of 13 zeros. This doesn't make much sense to me, since I would expect to see a length. (With value 2.)
Finally, I see characters "a" and "b". (Which is the original data.)
kdb+ on-disk format is not documented and subject to change. kdb+ should be used to read/write its data.
A shared object for kdb+ can be written, if a 3rd party needs access to data (e.g. q script reads data from disk & calls c function with data, or vice-versa) , https://code.kx.com/q/interfaces/using-c-functions/ , or kdb+ can use IPC to send the data to/from a 3rd party.
The reason we are trying to do this is we have other processes written in other languages which we want to be able to read data written to disk by Q/KDB.
These are obscure languages, and there are no clients directly supported/provided by KDB (the organization).