What should scaling channels look like?

[TheSteveBurgess]

I've spent some time looking at the best way to scale channels and i've not found a huge amount of information. The information i have found looks to be old and potentially written for older versions of Channels.

If you have an application that could potentially have tens of thousands of users, what would be the correct way to scale channels to cope with this?

I'm currently using Openshift (which is essentially just Kubernetes). With Django itself I'd scale horizontally, creating more containers as demand ramps up and then just round robin load balance between them. I assume with the long running nature of websocket connections this won’t work as the connection needs to come back to the correct container?

I'd be interested to hear how other people have tackled the issue.

[carltongibson]

Hi!

@raphaelm gave a great talk at DjangoCon Europe 2023 on this topic: Building and scaling a live event platform with django-channels — I'd start there.

[carltongibson]

Please do follow-up with your findings! Everyone is learning the same lessons, so a blog post, or some other write up is really valuable.

[raphaelm]

We use a combination of openresty and haproxy for load balancing. openresty (which is a nginx distribution) for SSL certificate handling and some custom logic, haproxy for the actual load balancing. Both of them are rock-solid and, with the correct configuration, I have not yet seen them to be a bottleneck in almost anything. Still, we have multiple servers running with them which are all configured to be an DNS A-record for our domain, so they share the load (and we have a quick fix available if one of them goes down or needs maintenance by just removing it from DNS).

[TheSteveBurgess]

Do you know if the haproxy needs to use sticky sessions to keep a connection pinned to the correct instance or does channels have away of manging that internally?

(Great DjangoCon presentation by the way)

[carltongibson]

Hey @SteveBurgessStantec

I assume with the long running nature of websocket connections this won’t work as the connection needs to come back to the correct container?

I'm not sure I'm following your concern here. The same connection stays open for the duration. You'd only need stickiness if you need a second connection to hit the same container (but you shouldn't really, I'd guess). 🤔

[TheSteveBurgess]

Ah, of course, you're right! For some reason in my head i had the inbound connection potentially being routed to another container but the connection never actually closes so that's not going to happen. :)

Thanks for help and fast responses!