You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We often experience httpx.ConnectErrors when making API calls with the httpx async client, we can reproduce this by passing many (~1000s) of async requests to the client. Oddly it manifests as a nodename nor servname provided exception. Based on some experimentation, it looks like this might be related to max_keepalive_connections and keepalive_expiry. To rule out other culprits (load balancer, service itself) I'm running an equivalent test with aiohttp, which has no problem completing the same volume of requests. It's a different exception, but it seems comparable to experiences noted in #1171. Is there a recommended configuration or approach I should use here?
I have found that passing 10,000 GETs to a pretty bare-bones AsyncClient, to a variety of hosts, from my local machine achieves it every time. I've shared a full testing script below. I'am typically able to achieve a successful outcome by tweaking httpx.Limits to some ~extreme settings:
# default setting results in failures
limits = httpx.Limits(max_connections=100, max_keepalive_connections=20, keepalive_expiry=5.0)
# these settings achieve success
limits = httpx.Limits(max_keepalive_connections=10000, keepalive_expiry=30)
In the case of error, the (abridged) exception:
ValueError: 'example.com' does not appear to be an IPv4 or IPv6 address
...
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
...
The above exception was the direct cause of the following exception:
httpcore.ConnectError: [Errno 8] nodename nor servname provided, or not known
...
The above exception was the direct cause of the following exception:
httpx.ConnectError: [Errno 8] nodename nor servname provided, or not known
Full stack trace
Traceback (most recent call last):
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 192, in connect_tcp
addr_obj = ip_address(remote_host)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/oli/.pyenv/versions/3.11.3/lib/python3.11/ipaddress.py", line 54, in ip_address
raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
ValueError: 'example.com' does not appear to be an IPv4 or IPv6 address
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
yield
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 114, in connect_tcp
stream: anyio.abc.ByteStream = await anyio.connect_tcp(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 195, in connect_tcp
gai_res = await getaddrinfo(
^^^^^^^^^^^^^^^^^^
File "/Users/oli/.pyenv/versions/3.11.3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/oli/.pyenv/versions/3.11.3/lib/python3.11/socket.py", line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 67, in map_httpcore_exceptions
yield
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 371, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 268, in handle_async_request
raise exc
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 251, in handle_async_request
response = await connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
raise exc
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
stream = await self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 124, in _connect
stream = await self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp
return await self._backend.connect_tcp(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 112, in connect_tcp
with map_exceptions(exc_map):
File "/Users/oli/.pyenv/versions/3.11.3/lib/python3.11/contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 8] nodename nor servname provided, or not known
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/private/tmp/httpx-test/test.py", line 52, in <module>
counter = asyncio.run(arun(method, host, nrequests))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/oli/.pyenv/versions/3.11.3/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/Users/oli/.pyenv/versions/3.11.3/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/oli/.pyenv/versions/3.11.3/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/test.py", line 35, in arun
statuses = await run_httpx(endpoint, nrequests)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/test.py", line 24, in run_httpx
return [r.status_code for r in await asyncio.gather(*tasks)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/test.py", line 10, in get_url
res = await client.get(endpoint)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_client.py", line 1786, in get
return await self.request(
^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_client.py", line 1559, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_client.py", line 1646, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_client.py", line 1674, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_client.py", line 1711, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_client.py", line 1748, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 370, in handle_async_request
with map_httpcore_exceptions():
File "/Users/oli/.pyenv/versions/3.11.3/lib/python3.11/contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
File "/private/tmp/httpx-test/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 84, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 8] nodename nor servname provided, or not known
Testing script
import asyncio
import time
from collections import Counter
import aiohttp
import httpx
async def get_url(i, client, endpoint):
print(f"GET {i}", endpoint)
s = time.time()
res = await client.get(endpoint)
e = time.time()
print(i, (e-s), res)
return res
async def run_httpx(endpoint, nrequests):
# the default httpx settings
limits = httpx.Limits(max_connections=100, max_keepalive_connections=20, keepalive_expiry=5.0)
# configuration that allows this task to succeed
# limits = httpx.Limits(max_keepalive_connections=10000, keepalive_expiry=30)
timeout = httpx.Timeout(None)
async with httpx.AsyncClient(limits=limits, timeout=timeout) as client:
tasks = [get_url(i, client, endpoint) for i in range(nrequests)]
return [r.status_code for r in await asyncio.gather(*tasks)]
async def run_aiohttp(endpoint, nrequests):
async with aiohttp.ClientSession() as session:
tasks = [get_url(i, session, endpoint) for i in range(nrequests)]
return [r.status for r in await asyncio.gather(*tasks)]
async def arun(method, endpoint, nrequests):
c = Counter()
if method == 'httpx':
print('running httpx')
statuses = await run_httpx(endpoint, nrequests)
else:
print('running aiohttp')
statuses = await run_aiohttp(endpoint, nrequests)
for s in statuses:
c[s] += 1
return c
if __name__ == '__main__':
method = 'httpx'
host = 'https://www.example.com'
nrequests = 10_000
assert method in ['aiohttp', 'httpx']
s = time.time()
counter = asyncio.run(arun(method, host, nrequests))
print(counter.total(), counter)
print(f'took {time.time()-s}')
Versions & configuration
Hardare: Mac / M1 Pro
Service target: uvicorn/fastapi, behind an AWS ALB (load balancer)
Python: 3.11.3
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
We often experience
httpx.ConnectError
s when making API calls with the httpx async client, we can reproduce this by passing many (~1000s) of async requests to the client. Oddly it manifests as anodename nor servname provided
exception. Based on some experimentation, it looks like this might be related tomax_keepalive_connections
andkeepalive_expiry
. To rule out other culprits (load balancer, service itself) I'm running an equivalent test with aiohttp, which has no problem completing the same volume of requests. It's a different exception, but it seems comparable to experiences noted in #1171. Is there a recommended configuration or approach I should use here?I have found that passing 10,000 GETs to a pretty bare-bones AsyncClient, to a variety of hosts, from my local machine achieves it every time. I've shared a full testing script below. I'am typically able to achieve a successful outcome by tweaking
httpx.Limits
to some ~extreme settings:In the case of error, the (abridged) exception:
Full stack trace
Testing script
Versions & configuration
Hardare: Mac / M1 Pro
Service target: uvicorn/fastapi, behind an AWS ALB (load balancer)
Python: 3.11.3
Pip freeze:
Beta Was this translation helpful? Give feedback.
All reactions