Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-126434: Use RLock for multiprocessing Event to avoid deadlock when handle system signal #126437

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

ivarref
Copy link

@ivarref ivarref commented Nov 5, 2024

Please see issue #126434 for details about bug and code to reproduce deadlock.

Kind regards.

@ivarref ivarref requested a review from gpshead as a code owner November 5, 2024 11:03
Copy link

cpython-cla-bot bot commented Nov 5, 2024

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Nov 5, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

Copy link
Contributor

@Zheaoli Zheaoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM

BTW this PR title need be changed

@ivarref ivarref changed the title gh-126434: Fix multiprocessing.Event to use reentrant lock to support .set()… gh-126434: Use RLock for multiprocessing Event to avoid deadlock when handle system signal (#126437) Nov 5, 2024
@ivarref ivarref changed the title gh-126434: Use RLock for multiprocessing Event to avoid deadlock when handle system signal (#126437) gh-126434: Use RLock for multiprocessing Event to avoid deadlock when handle system signal Nov 5, 2024
@ivarref
Copy link
Author

ivarref commented Nov 5, 2024

Mostly LGTM

BTW this PR title need be changed

I've changed the title @Zheaoli . What do you think? Thanks again.

Copy link
Contributor

@Zheaoli Zheaoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -0,0 +1 @@
Use RLock for :mod:`multiprocessing` ``Event`` to avoid deadlock when handle system signal
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use RLock for :mod:`multiprocessing` ``Event`` to avoid deadlock when handle system signal
Use :class:`multiprocessing.RLock` for :class:`multiprocessing.Event` to avoid deadlock when handle system signal

@Zheaoli
Copy link
Contributor

Zheaoli commented Nov 5, 2024

This patch is failed in free threading TSAN test. need to figure out the root cause

@tomasr8
Copy link
Member

tomasr8 commented Nov 5, 2024

This test seems to fail in other PRs as well, I restarted the job again to see if it fails again.

Copy link
Contributor

@colesbury colesbury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Event implementation is not correct with an RLock because that allows reentrancy (from the same thread) during operations that are not written to be reentrant. For example:

    def is_set(self):
        with self._cond:
            if self._flag.acquire(False):
                self._flag.release()
                return True
            return False

After the self._flag.acquire(False) before the self._flag.release(), reentrant is_set() calls will incorrectly appear to be false.

Event is not written to be safely called from a Python signal handler.

@bedevere-app
Copy link

bedevere-app bot commented Nov 5, 2024

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@Zheaoli
Copy link
Contributor

Zheaoli commented Nov 5, 2024

Event is not written to be safely called from a Python signal handler.

If this is the origin design behavior. Should we update the docs?

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a test, anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants