I asked this same question in StackExchange, but looks like this is more specific to KDB so posting it here.
|
|
I need to update a KDB table with new/updated/deleted rows while it is being read by other threads. Since writing to K structures while other threads access will not be thread safe, the only way I can think of is to clone the whole table and apply new changes to that. Even to do that, I need to first clone the table, then find a way to insert/update/delete rows from it. I’d like to know if there are functions in C to: 1. Clone the whole table 2. Delete existing rows 3. Insert new rows easily 4. Update existing rows
Appreciate suggestions on new approaches to the same problem as well.
|
assuming linux utilizing /tmp as an in mem file system
use 2 processes
splay the table in e.g. /tmp/t
map the table by process A - these are your readers
map the table by process B - this is your writer; apply updates (it’s cow) and resave it as the new splay (t1), msg to process A to load that new splay.
process A maps new splay when ready, removes old splay
no need for c, but if you like just use k(0,“q code here”,(K)0) to call q code to do all the deletes,updates,etc.
k(0,“q code here”,(K)0)
Does this call run the Q command inside the current process?
I tried this but get -128 in the return value always. (with a valid Q command).
On Thursday, 5 June 2014 15:21:11 UTC+1, Charles Skelton wrote:
assuming linux utilizing /tmp as an in mem file system
use 2 processes
splay the table in e.g. /tmp/t
map the table by process A - these are your readers
map the table by process B - this is your writer; apply updates (it’s cow) and resave it as the new splay (t1), msg to process A to load that new splay.
process A maps new splay when ready, removes old splay
no need for c, but if you like just use k(0,“q code here”,(K)0) to call q code to do all the deletes,updates,etc.
On Thu, Jun 5, 2014 at 3:50 PM, <kdb…@gmail.com> wrote:
I asked this same question in StackExchange, but looks like this is more specific to KDB so posting it here.
|
|
I need to update a KDB table with new/updated/deleted rows while it is being read by other threads. Since writing to K structures while other threads access will not be thread safe, the only way I can think of is to clone the whole table and apply new changes to that. Even to do that, I need to first clone the table, then find a way to insert/update/delete rows from it. I’d like to know if there are functions in C to: 1. Clone the whole table 2. Delete existing rows 3. Insert new rows easily 4. Update existing rows
Appreciate suggestions on new approaches to the same problem as well.
|
–
Submitted via Google Groups
yes. both of these are equivalent
K g(K x){return k(0,“value”,kp(“0N!`hello”),(K)0);}
K g(K x){return k(0,“0N!`hello”,(K)0);}
if you get -128, what error string do you see (in r->s)?
e.g.
K g(K x){K r=k(0,“`a+0”,(K)0);if(r->t==-128)O(“error %s\n”,r->s);R 0;}
should see
q)\l l32/a.q
`a
0
error type
Thanks for the reply.
This is the statement:
K kdb_table = k(0, “(a:symbol$();b:long$())”, (K)0);
And I get -128 in return.
printf(“error string: %s\n”, kdb_table->s);
And this prints ‘null’.
Do I need to any initialization work before kalling k()? I call khp(“”, -1); at the start of the program.
On Friday, 6 June 2014 10:49:53 UTC+1, Charles Skelton wrote:
e.g.
K g(K x){K r=k(0,“`a+0”,(K)0);if(r->t==-128)O(“error %s\n”,r->s);R 0;}
should see
q)\l l32/a.q
`a
0
error type
On Fri, Jun 6, 2014 at 11:46 AM, Charles Skelton <cha…@kx.com> wrote:
yes. both of these are equivalent
K g(K x){return k(0,“value”,kp(“0N!`hello”),(K)0);}
K g(K x){return k(0,“0N!`hello”,(K)0);}
if you get -128, what error string do you see (in r->s)?
On Fri, Jun 6, 2014 at 11:27 AM, <kdb…@gmail.com> wrote:
k(0,“q code here”,(K)0)
Does this call run the Q command inside the current process?
I tried this but get -128 in the return value always. (with a valid Q command).
On Thursday, 5 June 2014 15:21:11 UTC+1, Charles Skelton wrote:
assuming linux utilizing /tmp as an in mem file system
use 2 processes
splay the table in e.g. /tmp/t
map the table by process A - these are your readers
map the table by process B - this is your writer; apply updates (it’s cow) and resave it as the new splay (t1), msg to process A to load that new splay.
process A maps new splay when ready, removes old splay
no need for c, but if you like just use k(0,“q code here”,(K)0) to call q code to do all the deletes,updates,etc.
On Thu, Jun 5, 2014 at 3:50 PM, <kdb…@gmail.com> wrote:
I asked this same question in StackExchange, but looks like this is more specific to KDB so posting it here.
|
|
I need to update a KDB table with new/updated/deleted rows while it is being read by other threads. Since writing to K structures while other threads access will not be thread safe, the only way I can think of is to clone the whole table and apply new changes to that. Even to do that, I need to first clone the table, then find a way to insert/update/delete rows from it. I’d like to know if there are functions in C to: 1. Clone the whole table 2. Delete existing rows 3. Insert new rows easily 4. Update existing rows
Appreciate suggestions on new approaches to the same problem as well.
|
--
Submitted via Google Groups
that works for me.
q)\l l32/a.q
+ab!(symbol$();long$())
this is to create a shared library to be loaded into kdb+, not a standalone app that links with c.o.
i.e.
gcc -m32 a.c -o a.so -DKXVER=3 -shared -fPIC
(c.o does not contain any q language interpreter code.)
OK.
So your suggesstion is to compile my code to shared library and load that in to KDB+?
Unfortunately it might not work for me. Is there anything that I can do to have the Q interpreter loaded to my application instead?
Thanks
arles Skelton wrote:
that works for me.
q)\l l32/a.q
+ab!(symbol$();long$())
this is to create a shared library to be loaded into kdb+, not a standalone app that links with c.o.
i.e.
gcc -m32 a.c -o a.so -DKXVER=3 -shared -fPIC
(c.o does not contain any q language interpreter code.)
On Fri, Jun 6, 2014 at 12:16 PM, <kdb…@gmail.com> wrote:
Thanks for the reply.
This is the statement:
K kdb_table = k(0, “(a:symbol$();b:long$())”, (K)0);
And I get -128 in return.
printf(“error string: %s\n”, kdb_table->s);
And this prints ‘null’.
Do I need to any initialization work before kalling k()? I call khp(“”, -1); at the start of the program.
On Friday, 6 June 2014 10:49:53 UTC+1, Charles Skelton wrote:
e.g.
K g(K x){K r=k(0,“`a+0”,(K)0);if(r->t==-128)O(“error %s\n”,r->s);R 0;}
should see
q)\l l32/a.q
`a
0
error type
On Fri, Jun 6, 2014 at 11:46 AM, Charles Skelton <cha…@kx.com> wrote:
yes. both of these are equivalent
K g(K x){return k(0,“value”,kp(“0N!`hello”),(K)0);}
K g(K x){return k(0,“0N!`hello”,(K)0);}
if you get -128, what error string do you see (in r->s)?
On Fri, Jun 6, 2014 at 11:27 AM, <kdb…@gmail.com> wrote:
k(0,“q code here”,(K)0)
Does this call run the Q command inside the current process?
I tried this but get -128 in the return value always. (with a valid Q command).
On Thursday, 5 June 2014 15:21:11 UTC+1, Charles Skelton wrote:
assuming linux utilizing /tmp as an in mem file system
use 2 processes
splay the table in e.g. /tmp/t
map the table by process A - these are your readers
map the table by process B - this is your writer; apply updates (it’s cow) and resave it as the new splay (t1), msg to process A to load that new splay.
process A maps new splay when ready, removes old splay
no need for c, but if you like just use k(0,“q code here”,(K)0) to call q code to do all the deletes,updates,etc.
On Thu, Jun 5, 2014 at 3:50 PM, <kdb…@gmail.com> wrote:
I asked this same question in StackExchange, but looks like this is more specific to KDB so posting it here.
|
|
I need to update a KDB table with new/updated/deleted rows while it is being read by other threads. Since writing to K structures while other threads access will not be thread safe, the only way I can think of is to clone the whole table and apply new changes to that. Even to do that, I need to first clone the table, then find a way to insert/update/delete rows from it. I’d like to know if there are functions in C to: 1. Clone the whole table 2. Delete existing rows 3. Insert new rows easily 4. Update existing rows
Appreciate suggestions on new approaches to the same problem as well.
|
--
Submitted via Google Groups