threads started by librdkafka need to be able to set CPU affinity

[marlowa]

Maybe tinycthread could be extended to provide portable (i.e. Windows and linux) functions to set CPU affinity for threads started by librdkafka. This could be controlled by a suitable global config parameter. This is important in certain applications. This includes applications that can have a thread spin while waiting for an event. The idea is that spinning avoids a context switch and the thread is tied to a particular CPU to minimise the effect on other threads. I am working on such applications right now and it has been observed that some of these spinning threads are allocated a CPU which librdkafka threads also use. It was spotted during performance analysis. This can't be overridden in librdkafka because there is no config to do this and no associated CPU affinity code in tinycthread. The kafka thread thus has an adverse effect on whatever thread is tied to the CPU that librdkafka happens to get allocated. When the thread spins it also has an adverse effect on the kafka thread.

Some API would have to be arrived at that provides a portable way to express CPU affinity. Is it only Windows and linux that need to be catered for? What other platforms need to be taken into account?

[marlowa]

I've done a bit more thinking about this and it is harder than I thought. Even if it turns out to possible/easy to address all the platform portability issues in librdkafka to offer CPU affinity configuration, for reasons of symmetry it should also be offered for other language bindings. And there is no standard way in java. One would have to use a third party library such as java-thread-affinity, see https://github.com/OpenHFT/Java-Thread-Affinity. I couldn't even find a JEP to address this need. So kafka would end up depending on this java library. And what about other languages such as rust? These issues don't rule it out, it just shows that meeting the need is more than simply extending tinycthread and adding the config parameter. Let's discuss.

[fred-ro]

Pthreads affinity mask is inherited from it's creator thread. To achieve what you want, you can just create a thread with the proper affinity mask that will be used to interact with libRdKafka, thus making sure that any threads created there has the proper affinity.

There are also interceptors in libRdKafka. You can use on_thread_start interceptor to change current thread CPU affinity.