Ordered shutdown hooks for dependent threadpools

[mourjo]

Hi! I wanted to ask, do you think it would be a good idea to have a lifecycle manager (a shutdown hook) for multiple claypoole threadpools for when the JVM gets a kill signal? Something like if threadpool t1 uses another threadpool t2, then at the JVM shutdown, we would want the threadpool t2 to shutdown first in order for t1 to be gracefully shut down without having tasks rejected? We could have one shutdown hook for all claypoole threads in a JVM and based on the dependency (which I guess needs to be mentioned at the time of creation of the pool), it could coordinate shutting down of the different pools. Do you think this would be something worth having in this project? I could start a pull request then. Wanted to get your opinion first.

https://clojureverse.org/t/is-there-a-canonical-way-or-framework-to-handle-threads-lifecycle/6136

[leon-barrett]

Hi there! Sorry for the delay.

I like the idea of managed shutdown. In general, however, the dependency graph might be cyclic, e.g.:

(def http-pool (cp/threadpool))
(def db-pool (cp/threadpool))

(def results
  (->> items
    (cp/pmap http-pool http-request-1)
    (cp/pmap db-pool fetch-db-data)
    (cp/pmap http-pool http-request-2)))

Also, I do think something like https://github.com/stuartsierra/component is generally a better fit than exit-hook magic. The main advantage of Component is avoiding global mutable state, and I think that would be helpful here.

That said, if you have an API proposal, I'd be happy to consider it. The following considerations seem relevant:

• I'd like to avoid making it a default, because that's too magical. I'd prefer an explicit function like (add-shutdown-hook my-threadpool :after other-threadpool).
• I'd like to avoid dependencies on other libraries. In general, reuse is good, but I think threadpools are such a foundational element that I'd like to keep Claypoole dependency-free.

Another option that might make sense is just to take your graceful shutdown article https://medium.com/helpshift-engineering/achieving-graceful-restarts-of-clojure-services-b3a3b9c1d60d and make it a library that composes nicely with Claypoole. (It didn't look like it was a published library yet, but I wasn't sure.) The advantage there is that you can make something more general-purpose, and it might help meet that need for more cases than just Claypoole threadpools.

Thanks for the idea, and let me know how you want to follow up.

[mourjo]

Hey thanks for taking the time to reply in detail! Let me chalk up something, I like the idea of an independent project which can play well with shutting down threadpools in general. Initially, I was also thinking of a DAG-like approach (like namespaces which can't depend on each other), so in that respect, the :after API sounds good). What I'll do is try to implement something like this in a separate repository and share it here maybe. Or maybe it's better to close this issue. Thanks for the help though, appreciate it!

[ieugen]

Hi @mourjo ,

Do you think this issue is still relevant / something worth pursuing ?

Leon does not work with Clojure and we managed to move the project here (clj-commons) and I stepped in to maintain it.
The project is currently feature complete and we don't expect much change to happen besides maintenance and docs improvements.

I haven't had a chance to read through your proposal yet.
I decided to ask your thoughts on it first.
Please reach out with anything related to claypoole.

Thanks,
Eugen