Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support wildcards in MQTT topic filters matching retained messages #8824

Open
ansd opened this issue Jul 10, 2023 · 8 comments
Open

Support wildcards in MQTT topic filters matching retained messages #8824

ansd opened this issue Jul 10, 2023 · 8 comments

Comments

@ansd
Copy link
Member

ansd commented Jul 10, 2023

MQTT spec:

the Server MUST send the retained messages matching the Topic Filter of the subscription to the Client

However, RabbitMQ does not send retained messages for Topic Filters containing wildcards.

The solution should comply with at least the following requirements:

  1. The solution should be (somewhat) efficient. For example if there are 100k retained messages, if a new client subscribes, the solution must not perform a full table scan over all 100k retained messages/topics.
  2. The solution must not overload the broker. For example if there are 100k retained messages, and a client subscribes with a topic filter matching 50k of these retained messages, the broker must not send all 50k messages in one go.
@robertsLando
Copy link

A future RabbitMQ release will replicate retained messages in the cluster and also send retained messages matching topic filters containing wildcards.

Does this issue also takes track also of the first part about the retained messages that are not replicated in the cluster? In my case this is a big limitation, is there any work in progress or ETA for that?

@ansd
Copy link
Member Author

ansd commented Feb 28, 2024

Does this issue also takes track also of the first part about the retained messages that are not replicated in the cluster?

No, this is tracked in #8096

is there any work in progress or ETA for that?

No. For the time being you can write your own plugin as for example done in https://github.com/gery0815/rabbitmq-mqtt-retained-msg-redis

@robertsLando
Copy link

@ansd Thanks for that! Is there any mongodb alternative? Otherwise we could create one

cc @getlarge

@ansd
Copy link
Member Author

ansd commented Feb 28, 2024

Is there any mongodb alternative?

I'm not aware of a mongodb plugin.

@juergen-albert
Copy link

Hi, we ran into the same issue. Even though rabbitmq is our preferred broker, we can't use it because of this issue. We unfortunately we don't have anybody knowing anything about writing erlang. Thus we can't provide a PR for this.

We are willing to financially support anybody, who is willing to fix this. Who would be the right point of contact? @ansd?

@ansd
Copy link
Member Author

ansd commented Jul 9, 2024

@juergen-albert are you referring to only this issue or also to #8096?
If you are referring to only this issue, I assume you run only a single node RabbitMQ cluster?

@juergen-albert
Copy link

Right now, I refer only to this issue and yes we are currently run on a single node.

@ansd
Copy link
Member Author

ansd commented Jul 10, 2024

@juergen-albert please contact [email protected] to become a commercial VMware Tanzu RabbitMQ customer in order to discuss prioritisation of this feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants