Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add MessageQueue class #10

Merged
merged 3 commits into from
Jun 14, 2024
Merged

add MessageQueue class #10

merged 3 commits into from
Jun 14, 2024

Conversation

bkueng
Copy link
Contributor

@bkueng bkueng commented May 14, 2024

When using Connection.add_message_callback from python, there's a potential for deadlocks:

  • if the message receiving thread receives a message:
    • takes the _message_callback_mtx lock
    • calls back into python which takes the GIL lock
  • if the main thread calls Connection.expect (or receive):
    • GIL lock is held
    • takes the _message_callback_mtx lock So both threads take the locks in opposite order, which can result in deadlocks.

To prevent this we could make sure libmav does not hold locks when calling back to python, unlock GIL when calling Connection.expect (and others), or ensure the receive thread does not call back into python. This patch implements the last solution by introducing an (async) message queue. It allows python code to receive and handle messages at well-defined points.

bkueng and others added 3 commits June 13, 2024 13:57
When using Connection.add_message_callback from python, there's a potential
for deadlocks:
- if the message receiving thread receives a message:
  - takes the _message_callback_mtx lock
  - calls back into python which takes the GIL lock
- if the main thread calls Connection.expect (or receive):
  - GIL lock is held
  - takes the _message_callback_mtx lock
So both threads take the locks in opposite order, which can result in
deadlocks.

To prevent this we could make sure libmav does not hold locks
when calling back to python, unlock GIL when calling Connection.expect (and
others), or ensure the receive thread does not call back into python.
This patch implements the last solution by introducing an (async) message
queue. It allows python code to receive and handle messages at well-defined
points.
@ThomasDebrunner
Copy link
Contributor

Thanks, I rebased to main and added feature to use python iterators and len functions.

queue = libmav.MessageQueue(client_conn)

# wait a bit and do other work

print(f"Have {len(queue)} messages in queue")  # will have some messages in queue when received

for message in queue:
    print(message.name)

print(f"Have {len(queue)} messages in queue") # will be empty after iterating through.

@ThomasDebrunner ThomasDebrunner merged commit 3b8ed9f into main Jun 14, 2024
7 checks passed
@bkueng bkueng deleted the message_queue branch June 14, 2024 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants