Add sidekiq swarm integration

[fatkodima]

We use sidekiq enterprise, which has a swarm feature (https://github.com/sidekiq/sidekiq/wiki/Ent-Multi-Process).
It has a master process, which preloads the gems and application and forks child processes. When master preloads the gems, it also loads coverband. Coverband sets at_exit hook and a background thread (Coverband::Background) in the master. But when it forks the child processes, only the at_exit hook remains in the child, but the background thread is not working anymore (in the dead state).

I spent a good amount of time to figure out why we have some coverage missed, even though the code is clearly running in production.

I propose to add a simple integration for sidekiq swarm to make future people lifes easier.

Maybe it is also a good idea to make the solution more generic by somehow identifying when the process was forked and recreate the background thread? 🤔

[danmayer]

if this is not for the free or pro, but just swarm, can the integration file be sidekiq_swarm

[danmayer]

yeah, there are so many different servers, I have in the past tried to document how you need to handle before_fork and after_fork... the middleware design is intended to work generically across forking web servers... but we could look more at an approach like redis-client and fork detection....

https://github.com/redis-rb/redis-client/blob/43b894880252a91e0aa4de14b12d16af5a7b26d5/lib/redis_client/pid_cache.rb#L19

I think that coverband is a bit unique in that it needs to load and start the share library coverage before you load your other code, so it has the early initialization issue... but then it also has to deal with forking and starting a background thread... There are definitely ways to improve this and split out the shared lib coverage support from the worker thread and adapter connections... but that is probably a sizable refactoring.

[danmayer]

thanks for the work @fatkodima take a look at my comments and let me know how you want to test things and verify, guessing an RC release, but you could also point to your fork and verify things work and let me know

[fatkodima]

Changed to sidekiq_swarm.

I was able to replicate the problem locally and pointed the app to the local changes of this gem and it works.