I’m trying to figure out how to pass primitive arrays (most commonly, byte arrays) back and forth across c/q. Specifically, I will to call an external library that wants to operate on lists as C-arrays.
To that end, a few obvious things I haven’t figured out:
How do I query the length of a list?
How are these lists stored in kdb? I’d thought someone told me that kdb optimizes access by internally representing primitive lists as C-style arrays and accessing via pointer arithmetic.
> I’m trying to figure out how to pass primitive arrays (most commonly, byte arrays) back and forth across c/q. Specifically, I will to call an external library that wants to operate on lists as C-arrays.
How about we define a function that returns a reverse of a byte array:
K reverse(K x)
{K r; //the return value
if(x->t!=KG)return krr(“type”); //check type is array of bytes
r=ktn(KG,x->n); //create a new array of bytes with length the same as x
DO(x->n,kG(r)[i]=kG(x)[(n-1)-i]) //DO defined in k.h, kG is accessor for byte array
return r;
}
> How do I query the length of a list?
list->n
> How are these lists stored in kdb? I’d thought someone told me that kdb optimizes access by internally representing primitive lists as C-style arrays and accessing via pointer arithmetic.
yes - see the struct defined in k.h. the n field is the number of elements. the G0 element is the start of the array.
On Sunday, May 29, 2016 at 8:34:13 PM UTC-4, effbiae wrote:
How about we define a function that returns a reverse of a byte array:
K reverse(K x)
{K r; //the return value
if(x->t!=KG)return krr(“type”); //check type is array of bytes
r=ktn(KG,x->n); //create a new array of bytes with length the same as x
DO(x->n,kG(r)[i]=kG(x)[(n-1)-i]) //DO defined in k.h, kG is accessor for byte array
return r;
}
> How are these lists stored in kdb? I’d thought someone told me that kdb optimizes access by internally representing primitive lists as C-style arrays and accessing via pointer arithmetic.
yes - see the struct defined in k.h. the n field is the number of elements. the G0 element is the start of the array.
Thanks, that example helps quite a bit and I like the DO definition. The internal struct wrapping fields n and G0 threw me off as I didn’t notice it was anonymous.
So it looks like kG(x) returns the pointer x->G0. If I have to pass that buffer of data to a C function as a void*, can I safely pass kG(x) as long as the function will not modify the buffer?
> So it looks like kG(x) returns the pointer x->G0. If I have to pass that buffer of data to a C function as a void*, can I safely pass kG(x) as long as the function will not modify the buffer?
Yes. It’s convenient that any simple list (byte, char, int, ..) in kdb+ is implemented as a simple array in x->G0. You can use or pass kG(x) as you like. You can even reduce x->n with expected results (and memory management will still work).
As you can see from the example, you can modify a K (after ktn), so you can pass a K object to a function that modifies it’s contents, but be careful when modifying any arguments passed to your K function - K’s policy is “pass by value” (copy on write).