How do you share a variable across multiple workers?

[yangh0597]

How do you share a variable across multiple workers? Different pid

[zacqed]

We use redis as the caching layer for being able to share it across

[yangh0597]

This object is a complex object, hundreds of megabytes in size

[tilgovi]

Use an external store. Gunicorn workers are designed to share nothing.

[yangh0597]

In the field of machine learning, it is necessary to share a few hundred megabytes of objects, and then provide prediction functions through the interface, is there any good solution?

[tilgovi]

Again, Gunicorn workers are designed to share nothing. The only way to share things is to use OS methods of memory sharing.

Gunicorn uses fork() without exec(), so Gunicorn workers share any memory that was allocated before the worker started. Using the preload option or putting code in your config module, you may be able to load Python modules and have (an initial snapshot of) their data shared between workers. Changes will not be reflected across workers, though, because that's not how OS memory management works! Be aware that doing this can be brittle. In particular, macOS has recently become quite strict about what's allowed after fork() without exec().

A common pattern for any web service that has heavy work is to submit that work to a queue and handle it with a separate service. You can have some lower memory machines serving web requests and higher memory machines performing the prediction work.

Consider whether you really need to run multiple prediction processes on the same machine. If you do, and you really care about de-duplicating their memory consumption, look into ways to share memory between those processes. Sharing Python objects is not easy. The multiprocessing module provides some ability to share ctypes objects, but not arbitrary Python objects. It can share arbitrary objects via proxies, but that's really just keeping the objects in one process. If you simply need to read a file, but are okay with building the Python objects in each process, consider just mmap.

[tilgovi]

Consider also whether maybe Python is not the best language for your memory-and-cpu-intensive work.