Pre ping failing with uvloop #966
Labels
Comments
|
same issue, in my case, pool_recycle=3600, mysql wait_timeout is 28800 Logs/tracebacks SQLAlchemy and here is a related pr |
|
Same issue +1, is there any fix for it now? Version List: The version of MySQL is 8.0.39. |
|
Can be fixed by not using uvloop but vanilla asyncio instead: In case one of your used packages depends on uvloop (which was the case for me) just add
before any async operation is started. An MRE has also been shared recently in case you want to reproduce the issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
When using
aiomysqlwithuvloop==0.17.0, whenever MySQL closes a connection (due towait_timeout), the subsequent pre-ping command fails with:This bug does not occur when
uvloopis not used. Not sure whether to raise the issue withuvloopinstead.Might be related to #195?
To Reproduce
uvloop==0.17.0anduvicorn==0.23.2.pre_ping=Trueandpool_recyclenot set.wait_timeout.Expected behavior
The pre-ping request should succeed, and the HTTP request should succeed as well.
Logs/tracebacks
ERROR [root] Error: unable to perform operation on <TCPTransport closed=True reading=False 0x62afa5bd73a0>; the handler is closed ERROR [root] Stack: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 97, in receive return self.receive_nowait() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 92, in receive_nowait raise WouldBlock anyio.WouldBlock During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 78, in call_next message = await recv_stream.receive() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 112, in receive raise EndOfStream anyio.EndOfStream During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/hisd_be4fe/api/errors.py", line 33, in catch_middleware_exceptions return await call_next(request) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 84, in call_next raise app_exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 70, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ await self.simple_response(scope, receive, send, request_headers=headers) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__ raise exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__ await self.app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__ raise e File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__ await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__ await route.handle(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app response = await func(request) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 231, in app solved_result = await solve_dependencies( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/dependencies/utils.py", line 622, in solve_dependencies solved = await call(**sub_values) ^^^^^^^^^^^^^^^^^^^^^^^^ File "<makefun-gen-2>", line 2, in current_user_dependency File "/usr/local/lib/python3.11/site-packages/[redacted]/authentication/authenticator.py", line 143, in current_user_dependency user, _ = await self._authenticate( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/[redacted]/authentication/authenticator.py", line 184, in _authenticate user = await strategy.read_token(token, user_manager, authorized, ignore_expired) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/[redacted]/authentication/strategy/db/strategy.py", line 30, in read_token access_token = await self.access_token_db.get_by_token(token, max_age, authorized, ignore_expired) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/[redacted]/[redacted]/access_token.py", line 76, in get_by_token results = await self.session.execute(statement) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/ext/asyncio/session.py", line 439, in execute result = await greenlet_spawn( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 190, in greenlet_spawn result = context.throw(*sys.exc_info()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2246, in execute return self._execute_internal( ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2131, in _execute_internal conn = self._connection_for_bind(bind) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1998, in _connection_for_bind return trans._connection_for_bind(engine, execution_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<string>", line 2, in _connection_for_bind File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go ret_value = fn(self, *arg, **kw) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1123, in _connection_for_bind conn = bind.connect() ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3264, in connect return self._connection_cls(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 145, in __init__ self._dbapi_connection = engine.raw_connection() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3288, in raw_connection return self.pool.connect() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 452, in connect return _ConnectionFairy._checkout(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1366, in _checkout with util.safe_reraise(): File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__ raise exc_value.with_traceback(exc_tb) File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1304, in _checkout result = pool._dialect._do_ping_w_event( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 708, in _do_ping_w_event return self.do_ping(dbapi_connection) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 171, in do_ping dbapi_connection.ping(False) File "/usr/local/lib/python3.11/site-packages/sqlalchemy/dialects/mysql/aiomysql.py", line 191, in ping return self.await_(self._connection.ping(reconnect)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 125, in await_only return current.driver.switch(awaitable) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 185, in greenlet_spawn value = await result ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/aiomysql/connection.py", line 493, in ping await self._execute_command(COMMAND.COM_PING, "") File "/usr/local/lib/python3.11/site-packages/aiomysql/connection.py", line 724, in _execute_command self._write_bytes(prelude + sql[:chunk_size - 1]) File "/usr/local/lib/python3.11/site-packages/aiomysql/connection.py", line 669, in _write_bytes return self._writer.write(data) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/streams.py", line 332, in write self._transport.write(data) File "uvloop/handles/stream.pyx", line 674, in uvloop.loop.UVStream.write File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive RuntimeError: unable to perform operation on <TCPTransport closed=True reading=False 0x62afa5bd73a0>; the handler is closed ERROR [root] Method: GET ERROR [root] Url: [redacted] ERROR [root] req_time: 2023-09-11 13:46:49.891968 ERROR [root] Error: unable to perform operation on <TCPTransport closed=True reading=False 0x62afa5bf29a0>; the handler is closed ERROR [root] Stack: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 97, in receive return self.receive_nowait() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 92, in receive_nowait raise WouldBlock anyio.WouldBlock During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 78, in call_next message = await recv_stream.receive() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 112, in receive raise EndOfStream anyio.EndOfStream During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/hisd_be4fe/api/errors.py", line 33, in catch_middleware_exceptions return await call_next(request) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 84, in call_next raise app_exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 70, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__ await self.simple_response(scope, receive, send, request_headers=headers) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__ raise exc File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__ await self.app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__ raise e File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__ await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__ await route.handle(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app response = await func(request) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 231, in app solved_result = await solve_dependencies( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi/dependencies/utils.py", line 622, in solve_dependencies solved = await call(**sub_values) ^^^^^^^^^^^^^^^^^^^^^^^^ File "<makefun-gen-2>", line 2, in current_user_dependency File "/usr/local/lib/python3.11/site-packages/[redacted]/authentication/authenticator.py", line 143, in current_user_dependency user, _ = await self._authenticate( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/[redacted]/authentication/authenticator.py", line 184, in _authenticate user = await strategy.read_token(token, user_manager, authorized, ignore_expired) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/[redacted]/authentication/strategy/db/strategy.py", line 30, in read_token access_token = await self.access_token_db.get_by_token(token, max_age, authorized, ignore_expired) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/[redacted]/filuta_uds/access_token.py", line 76, in get_by_token results = await self.session.execute(statement) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/ext/asyncio/session.py", line 439, in execute result = await greenlet_spawn( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 190, in greenlet_spawn result = context.throw(*sys.exc_info()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2246, in execute return self._execute_internal( ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2131, in _execute_internal conn = self._connection_for_bind(bind) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1998, in _connection_for_bind return trans._connection_for_bind(engine, execution_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<string>", line 2, in _connection_for_bind File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go ret_value = fn(self, *arg, **kw) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1123, in _connection_for_bind conn = bind.connect() ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3264, in connect return self._connection_cls(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 145, in __init__ self._dbapi_connection = engine.raw_connection() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3288, in raw_connection return self.pool.connect() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 452, in connect return _ConnectionFairy._checkout(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1366, in _checkout with util.safe_reraise(): File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__ raise exc_value.with_traceback(exc_tb) File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1304, in _checkout result = pool._dialect._do_ping_w_event( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 708, in _do_ping_w_event return self.do_ping(dbapi_connection) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 171, in do_ping dbapi_connection.ping(False) File "/usr/local/lib/python3.11/site-packages/sqlalchemy/dialects/mysql/aiomysql.py", line 191, in ping return self.await_(self._connection.ping(reconnect)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 125, in await_only return current.driver.switch(awaitable) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 185, in greenlet_spawn value = await result ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/aiomysql/connection.py", line 493, in ping await self._execute_command(COMMAND.COM_PING, "") File "/usr/local/lib/python3.11/site-packages/aiomysql/connection.py", line 724, in _execute_command self._write_bytes(prelude + sql[:chunk_size - 1]) File "/usr/local/lib/python3.11/site-packages/aiomysql/connection.py", line 669, in _write_bytes return self._writer.write(data) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/streams.py", line 332, in write self._transport.write(data) File "uvloop/handles/stream.pyx", line 674, in uvloop.loop.UVStream.write File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive RuntimeError: unable to perform operation on <TCPTransport closed=True reading=False 0x62afa5bf29a0>; the handler is closedPython Version
3.11.4aiomysql Version
0.2.0PyMySQL Version
1.0.3SQLAlchemy Version
2.0.19OS
Ubuntu 20.04.4 LTS
Database type and version
8.0.15Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: