Hi,
I’m currently working in C++ and wish to export data to tables in kdb, real time.
I believe this is possible, and as a start I must include a k.h header file. Can someone tell me which is the appropriate file, and where to get it, as I think some example files on the web are outdated (possibly)?
I’m using the 32 bit version of kdb.
best,
J.
Hi John,
You should just use the C interface to interact with kdb+ using C++. The header file is located in the svn repository: http://code.kx.com/svn/kx/kdb+/c/c/k.h.
If you are building a shared library, you will need to be aware of the name mangling that the C++ compiler will perform when linking. To avoid this
you will need to give C linkage to any functions that you will dynamically load using extern “C”.
The svn repository also contains object files that you might need if you are building an executable or a binary that will not be loaded into kdb+:
http://code.kx.com/svn/kx/kdb+/l32/ (for linux 32 bit)
http://code.kx.com/svn/kx/kdb+/w32/ (for windows 32 bit)
http://code.kx.com/svn/kx/kdb+/m32/ (for mac 32 bit)
You can find more information in these links:
http://code.kx.com/wiki/Cookbook/InterfacingWithC
http://code.kx.com/wiki/Cookbook/ExtendingWithC (for shared libraries)
Mark Rooney
Financial Software Developer
AquaQ Analytics
Hi Mark, thanks for your help on this. When I include this header file and compile, line 8 returns an error. The problem seems to rest with #ifndef KXVER, resulting in all lines from 9 onwards commented out in the header file. What is this KXVER? I actually tried a different header file see: code.kx.com/wsvn/code/kx/kdb%2B/c/c/k.h?rev=1442&peg=1442 This compiles, though I’m unable to open a connection using khpu. This may not be related though. So not sure if this is the correct header to use. Any advice? -regards J
Are you defining KXVER? The easiest way to do this is to add a define before you include the header in your source code. You can also do it by passing a flag to your
compiler but the syntax for this differs based on compiler & platform used (e.g. for gcc on Linux you would pass -DKXVER=3).
e.g
#define KXVER 3
#include “k.h”
I forgot to add that the KXVER check is there to handle a change to the k struct that was not backwards compatible. To quote the Kx Wiki and the error message
in the k.h header file:
“Note that the k struct changed with the release of v3.0, and if you are compiling using the c library (c.o/c.dll) stamped on or after 2012.06.25 you should ensure you use the correct k struct by defining KXVER accordingly.”
and
“Set KXVER=3 for kdb+3.0 or standalone c-api after 2011-04-20. Otherwise set KXVER=2”
This works. Thanks! Ie putting the #define KXVER=3 in main. With regard to my other prob, when trying to run int c=khpu(“localhost”,5001,“password”); This fails since it doesn’t like the types, "warning: deprecated conversion from string constant to ‘S {aka char*}’ [-Wwrite-strings] .cannot convert ’ complex int’ to ‘I {aka int}’ for argument 2 to ‘I khpu(S, I, S)’. Should I expect this to compile ok? Regards John
You will want to make sure you are using #define KXVER 3 and not #define KXVER=3 before including your header file (the syntax is #define token value, no equals is required). Those
warnings about strings shouldn’t be a problem (and shouldn’t stop compilation unless you have a flag like gcc’s -Werror enabled). If you want to get rid of the warnings, you can explicitly cast the values
to non-const ones.
e.g.
int c = khpu((S)“localhost”, 1234,(S)“username:password”);
Thanks
Sorry my bad. The error at compile is coming from: undefined reference to ‘khpu’ Are you sure we only need the k.h file. There’s a time stored example online that shows compiling from Linux command window, where some libraries are also included. So maybe i need to include some libraries no? Also, I see examples on the tomestored using C not C++. You think C++ should be ok. Ie the issue doesn’t rest here. Thanks for you help on this. -John
Yep, using C++ should be fine. If you are creating a standalone executable, you will need to link against the c.o file. If you are using
Linux you can find it here: http://code.kx.com/svn/kx/kdb+/l32/
The c.o object provides all the definitions of the functions in the header file. You will also need to link against pthreads to resolve the missing
symbols for sem_post, pthread_rwlock_init etc.
For example, if I have a file called main.cpp that contains:
#include <iostream>
#define KXVER 3
#include “k.h”
int main(int argc, char* argv)
{
int c = khpu((S)“localhost”, 5000, (S)“username:pass”);
std::cout << "c: " << c << std::endl;
return 0;
}
I can compile it with the following line on Linux:
g++ main.cpp c.o -lpthread
Thanks
Hi Mark, success! The 64 bit version of the c.o file I had to use. I appreciate your help with all this. Great job. Now for the hard bit…to do what I need to do. I will likely ask more questions later. Thanks. -John
Hi Mark, After successfully connecting I’ve been able to play around with examples creating K objects. However I’m unaware of how to send variables defined in C++ to the kdb server. The examples seem to focus on the other way around eg: For some table tab created on q, I can access this in C++, by: K tab1=k(handle,“tab”,(K)0); However, say I have an int defined in C++, how do I retrieve this int variable in kdb.? Thanks, John.
Hi,
You can send C++ data to kdb+ by wrapping it up as a K object and passing it as a parameter to a function.
e.g
// Send a single integer to a function called myfunc in kdb. The ki function
// converts the integer into the corresponding K object representation that
// is used by kdb+. There are other functions for the other kdb+ types that
// can be found on the wiki page.
K result = k(handle, “myfunc”, ki(10), (K) 0);
// You can send multiple arguments by just listing them out and finishing the
// list with a terminating (K) 0. The example below sends a float and an integer
// to the function myfuncb.
K result = k(handle, “myfuncb”, kf(3.14), ki(17), (K) 0);
You would then define the following two functions in kdb to handle the data:
myfunc:{ … }
myfuncb:{[x;y] …}
You can see more realistic examples of this in the commented out part of the following file (calling the insert function with data records):
http://code.kx.com/svn/kx/kdb+/c/c/c.c
Thanks
Hi Mark, Thank you, This is working great. Again much help appreciated. However, just when I thought I was home and dry. I’ve been testing this out on a simple project, and it works fine. However when I try to do the exact same on my C++ application which includes additional header files, build now returns an error. It does not like the inclusion of one of my headers , error: “Include/k.h:99:11: error expected identifier before ‘return’.” #define R return “Include/boost/type traits/detail/is men fun pointer impl.hpp:38:17: note: in expansion of macro ‘R’” template
Include/k.h:99:11: error: expected ‘>’ before ‘return’"
#define R return
Thanks
-John
The #define R return that is in the k.h file is causing the R in your template parameter list to be replaced with return (the pre-processor
just blindly replaces all instances of R). To fix it, just undefine R after including the k.h header file:
e.g
#define KXVER 3
#include “k.h”
#undef R
Fantastic! It works. I owe you! -j
Yes, this is correct. <o:p></o:p>
<o:p> </o:p>
k.h has more these kinds of macros and it is only a matter of time that you would have another clashes. Especially when using with boost library together.<o:p></o:p>
<o:p> </o:p>
I would suggest you to put ur own identifier before each macros defined in k.h.<o:p></o:p>
<o:p> </o:p>
Here is an example:<o:p></o:p>
<o:p> </o:p>
http://code.kx.com/wsvn/code/contrib/kuentang/c%2B%2B/kdb/k.h <o:p></o:p>
<o:p> </o:p>
<o:p> </o:p>
Kim<o:p></o:p>
<o:p> </o:p>
Von: personal-kdbplus@googlegroups.com [mailto:personal-kdbplus@googlegroups.com] Im Auftrag von mark.rooney@aquaq.co.uk
Gesendet: Freitag, 1. Mai 2015 23:05
An: personal-kdbplus@googlegroups.com
Betreff: [personal kdb+] Re: Interacting with C++<o:p></o:p>
<o:p> </o:p>
The #define R return that is in the k.h file is causing the R in your template parameter list to be replaced with return (the pre-processor
just blindly replaces all instances of R). To fix it, just undefine R after including the k.h header file:
e.g
#define KXVER 3
#include “k.h”
#undef R
Hi Mark/Kim,
Now that I have this up and running, my goal is to collect some data in C++ and transfer it to kdb in the form of a dictionary. This seems like the most sensible method.
Then in kdb I can insert the data onto a table. The goal is to have this real-time.
I have been able to do the simpler task of sending across atoms with type float, however generalising to a dictionary is more difficult.
On top of that I cannot create K string objects as I cannot get the ss function to work.
Here is example code of my attempt which is not correct. Hopefully it’s not too far away, and you could point out the errors (please):
//Define empty table in kdb
tbl:( col1:();col2:();col3:())
//A dictionary K object is made up of key object and value object
//Define key
K key=ktn(KS,2); //only need 3 values
string col1=“string1”; //created string variable col1
string col2=“string2”;
string col3=“string3”;
kS(key)[0]=ss(col1); //intern the string using ss, of course not correct
kS(key)[1]=ss(col2);
kS(key)[2]=ss(col3);
//Define value
K val=ktn(0,2);
kK(val)[0]=kf(price1); //price1 and price2 are floats
kK(val)[1]=kf(price2);
kK(val)[2]=ks(sym); //not sure what to do here again for a string intern first? how?
//Create the dictionary
K result=xD(xkey,val);
K result1=k(handle,“func”,result,(K)0); //I opened a handle to kdb in earlier post
//where func is defined in kdb below:
func:{[d] tbl insert(d[col1];d[col2];d[col3])}
Note: I’m not experienced in C++ as you can tell. I want to get the data across from C++ to kdb where I am more comfortable.
I was initially not going to create a dictionary, just pass each value across separately where each takes a parameter is func.
However, if I have a lot of values then there’ll be too many parameters, so a dictionary seems like the sensible thing to use.
Though, as I mentioned i also had problems with strings, and I think the examples online are always char types, so how to do it for strings?
again, any help appreciated.
thanks,
John
Note you must call khpu before generating any K data, and the very first call to khpu must not be concurrent to other khpu calls. To initialise memory without making a connection, use khp(“”,-1);
In your example
K val=ktn(0,2);
should be
K val=ktn(0,3);
same for key length.
K result=xD(xkey,val);
should be key not xkey.
You can create a char vector with kp(“some string here”). Also kpn(“hello world”,5);
ks(“hello world”) automatically interns the string.
Hi Charles et al, Thanks for your corrections. I have two problems it seems at this point. 1. ks(“hello world”) works, and this produces a symbol type in kdb. However the below for example won’t work: string sval=“hello”; ks(sval); //doesn’t like this ks(ss(sval)); // nor does it like this. I need to do this because my sval (string) will continually change. 2. Given I cannot get the string type to work I’ll revert to a char example which does seem to, to point out second problem, though difference here we pass the address of char variables? int handle=khpu((S)“localhost”,5002,(S)“password”); K key = ktn(KS,2); char buf1=‘a’; char buf2=‘b’; char* buff1=&buf1; char* buff2=&buf2; kS(key)[0]=ss(buff1); KS(key)[1]=ss(buff2); K val=ktn(0,2); kK(val)[0]=kf(24); kK(val)[1]=kf(34); K dict=xD(key,val); K result=k(handle,“func”,dict,(K)0); kclose(handle); // where in kdb: func={[d] z::d}; The problem is, the output for z which is almost fine gives jibberish (Unicode replacement character) for the key values, though the val values are fine. The type of z is 99h so it is ok that way. Any ideas appreciated. Thanks, John.
Hi Charles et al, Thanks for your corrections. I have two problems it seems at this point. 1. ks(“hello world”) works, and this produces a symbol type in kdb. However the below for example won’t work: string sval=“hello”; ks(sval); //doesn’t like this ks(ss(sval)); // nor does it like this. I need to do this because my sval (string) will continually change. 2. Given I cannot get the string type to work I’ll revert to a char example which does seem to, to point out second problem, though difference here we pass the address of char variables? int handle=khpu((S)“localhost”,5002,(S)“password”); K key = ktn(KS,2); char buf1=‘a’; char buf2=‘b’; char* buff1=&buf1; char* buff2=&buf2; kS(key)[0]=ss(buff1); KS(key)[1]=ss(buff2); K val=ktn(0,2); kK(val)[0]=kf(24); kK(val)[1]=kf(34); K dict=xD(key,val); K result=k(handle,“func”,dict,(K)0); kclose(handle); // where in kdb: func={[d] z::d}; The problem is, the output for z which is almost fine gives jibberish (Unicode replacement character) for the key values, though the val values are fine. The type of z is 99h so it is ok that way. Any ideas appreciated. Thanks, John.