Q to C compiler

``

Hi,

I’ve recently added to github a Q to C (so lib) compiler - https://github.com/quintanar401/q2c . It can compile any Q code (small adjustments may be required) into a C lib that can be loaded into Q and then the compiled functions will work as before. It also supports C-like extention to Q so you may tranform Q values into C values and back and call C functions, all this in a safe manner without memory leaks or core dumps.

It can be used to create proxy functions for C libraries (see examples) with no effort at all, also you can use it to hide/obfuscate your Q code or code critical functions in C. I should warn that the compiled Q functions will not run faster than Q funcs themselves, this is because Q calls are done via dot func (similar to .) and it has a significant overhead.

To give an idea of how C-DSL looks like here is an example of the auto generated proxy for mmap function:

{[x1;x2;x3;x4;x5;x6] c.void_p.res:mmap[c.void_p$c.ij$x1; c.size_t$x2; c.i$x3; c.i$x4; c.i$x5; c.off_t$x6]; if[-1=c.ij$c.res;'"mmap: ",.string.strerror[C.toK C.errno]]; (C.toK `c.ij$c.res; x2)}

Expressions like c.xx$ tranform Q values into C values (they check that the type is correct), variables like c.var are local C variables and C.toK function is used to create K values from C values (it is overloaded on C type). Also the expression -1=c.ij$c.res inside “if” is pure C - C-DSL supports almost all C operators + abs, not, neg funcs. The above function will be compiled into:

K shmemex_mman_mmap__q(K _x1,K _x2,K _x3,K _x4,K _x5,K _x6){ K__st[1]; int __stc=0; K__ v=(K)1; void* _c_res; J _vc_j1; I _vc_i2; I _vc_i3; I _vc_i4; J _vc_j5; J _vc_j6; _Bool _vc_bool1; __v=__ K2j_cast(&_vc_j1,r1(_x6)); if(0==__v) goto__return; r0(__v); // c.j$ __v=__K2i_cast(&_vc_i2,r1(_x5)); if(0==__v) goto __return; r0(__v); // c.i$<val> __v=__ K2i_cast(&_vc_i3,r1(_x4)); if(0==__v) goto__return; r0(__v); // c.i$ __v=__K2i_cast(&_vc_i4,r1(_x3)); if(0==__v) goto __return; r0(__v); // c.i$<val> __v=__ K2j_cast(&_vc_j5,r1(_x2)); if(0==__v) goto__return; r0(__v); // c.j$ __v=__K2j_cast(&_vc_j6,r1(_x1)); if(0==__v) goto __return; r0(__v); // c.j$<val> _c_res=mmap((void*)(shmemexc_ij)_vc_j6,(size_t)_vc_j5,_vc_i4,_vc_i3,_vc_i2,(off_t)_vc_j1); // mmap[<val>;<val>;_vc_i4;_vc_i3;_vc_i2;<val>] // c.res[()]::<val> - no rval, global // end of statement _vc_bool1=-1LL==(shmemexc_ij)_c_res; // =[-1LL;<val>] if(0==_vc_bool1) goto __l2; // if/while check condition __ v=ki(errno); if(0==__v) goto__return; // C.toK[C.errno] __v=shmemex_string_strerror__ q(__v); if(0==__v) goto __return; // .string.strerror[<val>] __v=kdot(r1(__consts[51]),knk(2,r1(__consts[127]),__v)); if(0==__v) goto __return; // ,["mmap: ";<val>] __st[__stc++]=__v; if(__v->t==-KS)__v=krr(__v->s); else if(__v->t==KC) __v=krr(sn(kC(__v),__v->n)); else__v=krr("stype"); goto __return; // raise an exception r0(__v); // end of statement __l2: __ v=kj((shmemexc_ij)_c_res); if(0==__v) goto__return; // C.toK[<val>] __v=__ enlist(knk(2,__v,r1(_x2))); if(0==__v) goto __return; // enlist[<val>;x2] goto__return; // return cmd __return: r0(_x1); r0(_x2); r0(_x3); r0(_x4); r0(_x5); r0(_x6); while(__stc>0) r0(__st[--__stc]); return __v; };

A bit verbose but all K vars are freed and all errors are checked.

I could check the compiler only with gcc 32/64 bit but the generated file doesn’t contain any unusual code and it will probably compile with other c compilers as well (Visual Studio will require different include files I guess).

WBR, Andrey Kozyrev.
 

Very cool. Will definitely check it out

Cheers

Ryan

This is awesome.  BTW, I love your work ever since I discovered your q debugger!

I wonder if you considered compiling q code to LLVM IR?

Thanks,

Yes, I did and made the result of q2b look like llvm IR for this reason. But I think llvm is hard to use (you need to compile it first) and then it is unlikely that someone will be compiling something from Q very often. Bindings to llvm and translation to IR require a lot of work and as I couldn’t find a useful usecase I didn’t do anything yet.

WBR,
Andrey Kozyrev.

On Wednesday, October 11, 2017 at 4:17:50 PM UTC-4, Andrew Kozyrev wrote:

 Bindings to llvm and translation to IR require a lot of work and as I couldn’t find a useful usecase I didn’t do anything yet.

Right, but I could use it in my “Julia for kdb+” project.