Hi,
I have a table in which I need to store unique 32 byte cryptographic signatures for each row. I receive the column inputs via JSON as a 64 byte hex string. What’s the best column definition for that data and how would I convert the string to that type? Are there any hex to byte vector and byte vector to hex routines built in to kdb?
Thanks for any help!
Cheers,
Donovan.
Returns a byte vector from hex string: “X”$2 cut “deadbeef” To do the inverse transform, you can just use ‘string’ Sean
I’d use a compound byte column to store crypto signatures, i.e. list of lists of bytes.
You can use these two functions to convert hex strings to byte vectors and the other way around:
hex2bin: { “X”$'2 cut x }
bin2hex: { “” sv string x }
Hope this helps.
Sean and Igor, very helpful, thank you very much!
One additional question: what is the storage overhead of a list of lists of bytes? I’m assuming that a list of GUID’s has less overhead, as the fixed length is built in to the data type?
If your table is splayed the overhead will be 8 bytes (sizeof(long)) per crypto signature, here’s why: http://code.kx.com/wiki/JB:KdbplusForMortals/splayed\_tables#1.2.7\_Splayed\_Tables\_with\_Compound\_Columns
There will be some performance degradation, too. You can use GUID’s, but since a GUID is 16 bytes long you will need to split a crypto hash into 2 halves and store them in two separate GUID columns (and combine them during retrieval). I doesn’t look like a very elegant solution to me unless you really need to save some space.
Thanks for the explanation Igor, much appreciated!
Yes, less overhead for GUIDs.
List of lists is structured as a two-level index in q. The first level serves as an index to the second level, where the latter stores the flatten list of all bytes. You may consider the first level data as storage overhead in this sense.