Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Syncvar and FEB Lock/Unlock In Different Threads #207

Conversation

insertinterestingnamehere
Copy link
Collaborator

Fixes #202.

As best I can tell, this function is only ever called in one place and the lock is managed by the caller there, so no unlock is needed. It's probably a leftover typo from some previous refactoring.

@insertinterestingnamehere insertinterestingnamehere marked this pull request as draft December 6, 2023 19:36
@insertinterestingnamehere insertinterestingnamehere marked this pull request as ready for review December 7, 2023 00:09
@insertinterestingnamehere
Copy link
Collaborator Author

Found the corresponding lock to this unlock. The issue wasn't that the unlock was unpaired it was that the unlock was getting called from a completely different thread.

The fix for that will be to either move the lock/unlock pairing either entirely into the called thread or entirely outside of it. I'm favoring locking outside of it here, but I'm not entirely sure of the performance implications of lock placement here so it'd be great to get a second opinion on that.

@insertinterestingnamehere
Copy link
Collaborator Author

Okay, moving the lock/unlock to the outer scope caused a deadlock with the nemesis/hwloc config, so apparently it has to go inside the called function.

@insertinterestingnamehere
Copy link
Collaborator Author

Hmm. Moving it inside may not work either under some circumstances even though that was working locally. Marking as WIP for now so I can decide what to do with it.

Copy link
Collaborator

@janciesko janciesko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@insertinterestingnamehere insertinterestingnamehere changed the title Remove Extra Mutex Unlock In Syncvar Implementation Fix Syncvar and FEB Lock/Unlock In Different Threads Feb 9, 2024
@insertinterestingnamehere insertinterestingnamehere marked this pull request as ready for review February 9, 2024 00:13
@insertinterestingnamehere
Copy link
Collaborator Author

Alright! I sorted through what was going on here. This idiom was using a pthread mutex as a way to notify a waiting external thread that a blocked call had completed. Thread sanitizer was complaining since paired calls to lock/unlock from different threads aren't supposed to happen. Fortunately this is a fairly straightforward application of a condition variable so I just switched to using one of those.

@insertinterestingnamehere
Copy link
Collaborator Author

Okay, latest CI runs are showing some failures with the distrib scheduler now. Investigating...

@insertinterestingnamehere insertinterestingnamehere marked this pull request as draft February 9, 2024 17:50
@insertinterestingnamehere
Copy link
Collaborator Author

I can't reproduce this locally. I'll set it down for now and circle back to it later.

@insertinterestingnamehere insertinterestingnamehere marked this pull request as ready for review March 15, 2024 18:30
@janciesko
Copy link
Collaborator

What's the status of this?

@insertinterestingnamehere
Copy link
Collaborator Author

This one's good to go now. It was causing some lingering issues in CI last time you looked at it, but I've fixed those since then.

@insertinterestingnamehere insertinterestingnamehere merged commit 4b54807 into sandialabs:release-1.20-pre Mar 27, 2024
176 of 297 checks passed
@insertinterestingnamehere insertinterestingnamehere deleted the syncvar_lock branch October 3, 2024 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unpaired Mutex Unlock In Syncvar
2 participants