Client API async inserts and flush

Community Support
1

Hi All,

I’m quite new to kdb and I’ve got question about inserts from client API. Let’s say I have lot of messages coming from some external system and I want to persist them to the table as quickly as possible (I’ll use Java API code, but i think this is pretty much same for all APIs):

public void onMessage(Message m) {

  //prepare data for insertion here

  conn.ks(insertExpression); //async insert

  conn.k(“”); //flush

}

The “ks” method returns very fast. All it does - just puts data to the socket and return. The “k” method with empty argument does flushing as the side effect, but it really slow. If i remove it there can be data in the socket buffer stuck for some time.

I want on the one hand not to slow insertion calling additional method for flushing. On the other hand I want data to be flushed.

What the approach to this problem? Is there any obvious solution i’m missing? I think of flushing from another stream, but I’m not sure if it will be thread safe to use the same connection object.

Thanks in advance,

Alex.

2

conn.ks(insertExpression) will block until all data has been written into the tcp stack.
conn.k(“”) will block until the remote instance has processed all pending msgs from this socket, and responded to this sync msg.

The apis do not queue data for sending at the application layer, they block until data has been written in full into the tcp stack, regardless of whether the msg is async or sync. kdb+ itself is different to these apis, as it queues data for sending, and then feeds it into the socket when tcp flow control allows.

You can perhaps increase the size of your tcp send (&receive) buffer, so that more data can be written before the write call blocks.