Concurrent requests take forever to be executed #3151
-
Hello, I am currently having a big issue with httpx. When sending many requests at the same time, they all literally freeze. I am on Mac m2, latest version, and a friend of mine, on Mac m3, latest version, has no issues. Here is my screenshot: It only happens for concurrent requests, and the more there are, the more the "freezing time" lasts, or even, at a point, timeouts. Here is my traceback, if it is useful:
|
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 6 replies
-
Why are you creating 100 clients and sending a single request with each? Instead you should use a single client to send hundreds of requests. It sounds like you're running into a simple network saturation, since you're creating each client separately each request is sent on a new connection instead of sharing an existing connection. As each client creates a new connection, you eventually run into a timeout error since establishing new connections takes time and usually there's an upper bound on the number of concurrent connections you can have. |
Beta Was this translation helpful? Give feedback.
I'm pretty sure that does not automatically create a session for you in httpx, I think you want something like this: