updating a kbd table from C (shared lib)

Community Support
21

Hi,

It’s probably optimal in terms of performance to use pipes on a Linux machine, but we used sockets instead of pipes for a few reasons:

  - Trying to keep the code simple : One of the goals was that the code should compile and run on both Linux and Windows, so we made use of the selectable socketpair available here: https://github.com/ncm/selectable-socketpair.
This made it easier to abstract over the different operating systems and is reliable well tested code. Keeping the code simple also allows people to focus on the interaction with the kdb+ api to see how it works rather than wading through
platform specific implementation details or having to bother with the differences between sockets vs pipes (e.g send/recv and read/write calls).

  - Sockets are not that much slower than pipes on Linux : If you look at the implementation of the socket pair in the library above, you will see that it uses the AC_UNIX/AC_LOCAL socket family on Linux.
This is intended for communicating between processes/threads on the same machine and cuts down on a fair bit of the overhead you would get when using AC_INET sockets. On Windows I believe that the
performance gap between named pipes and sockets is bigger, but you can’t just blindly replace the socket with a pipe because of the issue below.

  - It’s tricky to get pipes to work nicely with some functions on Windows : the implementations of pipes on Linux and Windows are very different, and in particular trying to use the select() function on a pipe fails.
This can make the sd1() callback hang if passed a file descriptor that belongs to a pipe. I think that the Windows implementation of select() will only work on the handles generated via WinSock (but I’m not 100%
sure on this).

I hope this answers your question!

Thanks

22

Hi Mark,

  I expect a pipe to be about 10% faster than a UNIX socket on OSX and Linux . This shouldn’t be much of a problem since other parts of the code will probably be on the critical path well before the performance delta becomes an issue, Also, your note on windows select() failing on a pipe justifies the option for sockets on portability alone.

  Just a note. If you'll be having multiple threads triggering the same callback function, a pipe will guarantee that the writes are atomic (if less than PIPE_BUF). The selectable-socketpair is using a STREAM socket, which is not guaranteed to be atomic and I assume could be problematic in this scenario. One could probably change it to use a DATAGRAM socket to get over this (though I haven’t tested). You probably knew this already, but might be useful for someone using the feedhandler tutorial as a template. :)


Again, thank you all for your answers. They have been most helpful.


Tiago