Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

asyncpg __del__ methods call asyncio APIs directly, which isn't guaranteed to work and indeed, doesn't work #376

Open
njsmith opened this issue Oct 13, 2018 · 1 comment

Comments

@njsmith
Copy link

njsmith commented Oct 13, 2018

asyncpg has several __del__ methods that call into asyncio APIs. But... you can't safely call into asyncio APIs from a __del__ method, because __del__ methods can be run in arbitrary threads, or at arbitrary moments when the loop's internal data structures are in an inconsistent state.

Discovered here: python-trio/trio-asyncio#44

Looking at the traceback in that issue, I suspect that the reason this hasn't been noticed before is that the __del__ method ultimately ends up calling loop.call_soon(...). And if you do that from another thread, then with regular asyncio, things will mostly seem to work – it won't actually wake up the loop the way call_soon_threadsafe would, but if the loop is still running then it will eventually get run, and the default call_soon will not explode or otherwise notice if it's called from the wrong thread. But in that issue, someone's using asyncpg with an alternative asyncio loop that fails when call_soon is called from the wrong thread, and this breaks asyncpg.

I guess asyncpg should go through all its __del__ methods and wrap them in a loop.call_soon_threadsafe?

@njsmith
Copy link
Author

njsmith commented Oct 13, 2018

I guess asyncpg should go through all its __del__ methods and wrap them in a loop.call_soon_threadsafe?

This is assuming we go with my proposal in bpo-34968 and guarantee that it's safe to use call_soon_threadsafe inside __del__ methods.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant