RabbitMQ 4.0.4

RabbitMQ 4.0.4 is a maintenance release in the 4.0.x release series.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users and those
who hold a valid commercial support license.

It is strongly recommended that you read 4.0 release notes
in detail if upgrading from a version prior to 4.0.0.

Minimum Supported Erlang Version

This release requires Erlang 26 and supports Erlang versions up to 27.1.x.
RabbitMQ and Erlang/OTP Compatibility Matrix has more details on
Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Broker

Bug Fixes

• In rare cases quorum queue could end up without an elected leader because
  chosen candidate replica was not verified for aliveness.

  Contributed by @Ayanda-D.

  GitHub issues: #12727, #10423, #12701

• Quorum queue follower replicas that have falled behind the leader could
  run into an exception after installing a snapshot.

  GitHub issue: #12635

• Clusters with a large number of streams could run into confusing timeout
  exceptions.

  GitHub issue: #12693

• Stream members could fail to start when their data directories had externally added files,
  for example, metadata of certain file systems.

  GitHub issue: #12688

• Fetching metrics of AMQP 1.0 connections could fail with an exception.

  GitHub issue: #12700

• Nodes using Khepri for schema data store now follow a set of rabbitmqctl reset procedures
  better aligned with those performed by nodes still using Mnesia.

  GitHub issue: #12763

Enhancements

• Policy changes are now periodicaly re-applied (only if necessary) to quorum queues.
  Quorum queues that did not have an online elected leader at the time
  of policy change would now eventually "pick up" the settings from that policy.

  Contributed by @LoisSotoLopez.

  GitHub issue: #12667

• Clusters with many streams and stream consumers will see a reduced per-stream CPU and network I/O
  footprint.

  GitHub issue: #12685

• Clusters now can optionally be tagged with key-value pairs (cluster tags). The tags will
  be reported by rabbitmq-diagnostics cluster_status and the GET /api/overview HTTP API endpoint.

  Note that the Prometheus scraper API endpoint intentionally omits them because this kind of
  metadata in Prometheus is considered to be deployment and not application metadata.

  The tags are configured using rabbitmq.conf:

  cluster_tags.environment = production
  
  cluster_tags.region = us-east
  cluster_tags.az = us-east-3

  Contributed by @SimonUnge.

  GitHub issue: #12552

• Nodes now can optionally be tagged with key-value pairs (node tags). The tags will
  be reported by rabbitmq-diagnostics status and the GET /api/overview HTTP API endpoint.

  Note that the Prometheus scraper API endpoint intentionally omits them because this kind of
  metadata in Prometheus is considered to be deployment and not application metadata.

  The tags are configured using rabbitmq.conf:

  nodes_tags.environment = production
  
  nodes_tags.region = us-east
  nodes_tags.az = us-east-3

  Contributed by @SimonUnge.

  GitHub issue: #12703

• When a max length limit is applied to a quorum queue with a larger backlog (e.g. millions of messages),
  the deletion of excess messages now carries a significantly more moderate spike in memory footprint
  of the queue.

  GitHub issue: #12608

CLI Tools

Bug Fixes

• rabbitmq-diagnostics check_if_any_deprecated_features_are_used now takes more deprecated features
  into account.

  GitHub issue: #12734, #12738

MQTT Plugin

Bug Fixes

• A message with expiration (TTL) set, that was published by an AMQP 0-9-1 publusher,
  could not be converted for an MQTT consumer.

  GitHub issue: #12711

• When x.509 (TLS) certificate-based authentication was used, two keys that controlled
  what SAN (Subject Alternative Name) fields were used to fetch client identity did not
  have any effect when used in rabbitmq.conf.

  Partially contributed by @janezturk.

  GitHub issue: #12618

Prometheus Plugin and Grafana Dashboards

Bug Fixes

• Tweaks for Grafana 11.3 compatibility.

  Contributed by @anhanhnguyen.

  GitHub issue: #12720

Management Plugin

Enhancements

• The endpoint that creates bindings now uses a much smaller HTTP request body
  size limit by default. Unlike the definition upload endpoint that accepts
  large definition documents, bindings do not need the generous multi-MiB limit.

  Note that the default HTTP request body size limit can be configured,
  for example, to reduce it across the board.

  GitHub issue: #12697

• Improved alignment of optional queue arguments on the queue declaration page.

  Contributed by @markus812498.

  GitHub issue: #12678

OAuth 2 Plugin

Bug Fixes

• When configuring multiple resource servers,
  additional_scopes_key was not taken into account, which means some scopes were not considered
  when making an authorization decision.

Contributed by @Hathoute.

GitHub issue: #12750

Debian Package

Enhancements

• The package now list Erlang 27.x as supported series.

  GitHub issue: #12603

RPM Package

Enhancements

• The package now list Erlang 27.x as supported series.

  GitHub issue: #12603

Dependency Changes

• osiris was upgraded to 1.8.4

Source Code Archives

To obtain source code of the entire distribution, please download the archive named rabbitmq-server-4.0.4.tar.xz
instead of the source tarball produced by GitHub.

RabbitMQ 4.1.0-beta.1

RabbitMQ 4.1.0-beta.1 is the first preview of the next feature release.

Highlights

AMQP 1.0 Filter Expressions

As of #12415, RabbitMQ implements properties and application-properties filters of AMQP Filter Expressions Version 1.0 Working Draft 09 when consuming from a stream via AMQP 1.0.
String prefix and suffix matching is also supported.

This feature:

• adds the ability to RabbitMQ to have multiple concurrent clients each consuming only a subset of messages while maintaining message order, and
• reduces network traffic between RabbitMQ and clients by only dispatching those messages that the clients are actually interested in.

Support for Multiple Routing Keys in AMQP 1.0 via x-cc Message Annotation

PR #12559 enables AMQP 1.0 publishers to set multiple routing keys by using the x-cc message annotation.
This annotation allows publishers to specify a list of routing keys (strings) for more flexible message distribution, similar to the CC header in AMQP 0.9.1.

OAuth 2.0 Token Renewal on AMQP 1.0 Connections

PR #12599 introduces support for OAuth 2.0 token renewal on AMQP 1.0 connections.
This feature allows clients to set a new token proactively before the current one expires, ensuring uninterrupted connectivity.
If a client does not set a new token before the existing one expires, RabbitMQ will automatically close the AMQP 1.0 connection.

Metrics for AMQP 1.0 Connections

PR #12638 exposes the following AMQP 1.0 connection metrics in the RabbitMQ Management UI and the /metrics/per-object Prometheus endpoint:

• Bytes received and sent
• Reductions
• Garbage collections
• Number of channels/sessions

These metrics have already been emitted for AMQP 0.9.1 connections prior to RabbitMQ 4.1.

AMQP 1.0 Sessions and Links in the Management UI

PR #12670 displays detailed AMQP 1.0 session and link information on the Connection page of the Management UI including:

• Link names
• Link target and source addresses
• Link flow control state
• Session flow control state
• Number of unconfirmed and unacknowledged messages

Prometheus Histogram for Message Sizes

PR #12342 exposes a Prometheus histogram for message sizes received by RabbitMQ.

This feature allows operators to gain insights into the message sizes being published to RabbitMQ, such as average message size, number of messages per pre-defined bucket (which can both be computed accurately), and percentiles (which will be approximated).
Each metric is labelled by protocol (AMQP 1.0, AMQP 0.9.1, MQTT 5.0, MQTT 3.1.1, and MQTT 3.1).

Feature flags quality of life improvements

The introduction of required feature flags several minor versions ago showed the poor user experience around them. Therefore, several improvements were made to the subsystem and the management UI to improve the general usage:

• Required feature flags have now a soft/hard requirement attribute.

  Hard required feature flags are the ones already in use: the user has to enable a feature flag before upgrading to a version that requires it, otherwise the node will refuse to start.

  Soft required feature flags are the new kind: when the user upgrades to a version that requires a feature flag that is not enabled yet, the feature flag will be enabled automatically during startup. To achieve that, some compatibility code is kept with a soft required feature flag, unlike a hard required one. In the future, RabbitMQ will use soft required feature flags as much as possible. This is only a measure to help users that did not follow recommendations. The recommendations is still that feature flags should always be enabled at the best time for the workload.

  See #12466.

• The management UI now shows if a feature flag has a migration function (in other words, it may take time to be enabled), if it is experimental and whether it is supported or not. To enable an experimental feature flag, a user has to tick checkboxes to confirm they know what they are doing. The feature flags UI has other fixes under the hood; the most important one is that a feature flag that takes time to be enabled will not freeze the browser tab anymore (the HTTP request was synchronous and executed from the browser main thread before). See #12643.

• Required feature flags are hidden from the CLI and the management UI because there is nothing a user can do about them. See #12447.

• Logging was made less verbose. See #12444.

See the full GitHub project for the complete list of improvements and fixes.

Potential incompatibilities

• The default MQTT Maximum Packet Size changed from 256 MiB to 16 MiB. This default can be overridden by configuring mqtt.max_packet_size_authenticated. Note that this value must not be greater than max_message_size (which also defaults to 16 MiB).

RabbitMQ 4.0.3

RabbitMQ 4.0.3 is a maintenance release in the 4.0.x release series.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users and those
who hold a valid commercial support license.

It is strongly recommended that you read 4.0 release notes
in detail if upgrading from a version prior to 4.0.0.

Minimum Supported Erlang Version

This release requires Erlang 26 and supports Erlang versions up to 26.2.x.
RabbitMQ and Erlang/OTP Compatibility Matrix has more details on
Erlang version requirements for RabbitMQ.

Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Broker

Bug Fixes

• Classic queues could run into an exception.

  Kudos to @netrmqdev for helping the core team reproduce this rare behavior.

  GitHub issue: #12367

• Continuous membership reconciliation of quorum queues did not propagate a timeout error.

  Contributed by @SimonUnge.

  GitHub issue: #12578

• Quorum queues could truncate the log too aggresively (by one entry too many).

  GitHub issue: #12358

• Quorum queues failed to requeue a message with a specific workload where consumers
  requeued a delivery and then immediately cancelled themselves.

  GitHub issue: #12442

• When a quorum queue was forced to shrink, it did not stop the replicas on the nodes that were
  removed from the list of replicas. In many cases this had no visible effects because the node
  in question is stopped or even removed entirely from the cluster.

  Contributed by @Ayanda-D.

  GitHub issue: #12475

• AMQP 1.0 implementation now complies with the Anonymous Terminus extension (section 2.2.2 Routing Errors).

  GitHub issue: #12397

• For AMQP 1.0 clients, correct (compatible, sensible) combinations of the settle mode and a transfer's settled field
  are now enforced.

  GitHub issue: #12371

• If an AMQP 1.0 client used a reserved annotation key, the connection was closed
  with an exception.

  GitHub issue: #12527

• Messages with arrays in annotations published by AMQP 1.0 publishers and consumed by AMQP 0-9-1 consumers
  lead to an exception.

  GitHub issue: #12572

• Quorum queues with a configured delivery limit could run into an exception.

  GitHub issue: #12405

• Publisher ID length is now validated to not exceed its internal limit of 255 bytes.

  GitHub issue: #12499

Enhancements

• Initial support for Erlang/OTP 27, starting with 27.1.2.

  Releases prior to 27.1.2 are affected
  by several bugs that can seriously affect RabbitMQ users, in particular those using TLS for client connections.

  RPM and Debian packages will reflect Erlang 27 support in their metadata starting with a later patch release, 4.0.4.

  GitHub issue: #12208 (and many others, including on the Erlang/OTP side)

• Delivery requeue history is now better tracked using AMQP 1.0's Modified Outcome feature.

  GitHub issue: #12506

• Nodes now avoid logging potentially confusing messages about schema data store operations when
  querying for traces of any deprecated (or removed) features in the system.

  GitHub issue: #12348

Prometheus Plugin

Bug Fixes

• rabbitmq_queue_exchange_messages_published_total included a duplicate vhost label.

  Contributed by @LoisSotoLopez.

  GitHub issue: #12347

Management Plugin

Bug Fixes

• GET /api/queues/{vhost} and similar endpoints ran into an exception when a sorting parameter was provided and one of the
  queues in the result set was a quorum one.

  GitHub issue: #12374

Dependency Changes

• CSV was upgraded to 3.2.1

Source Code Archives

To obtain source code of the entire distribution, please download the archive named rabbitmq-server-4.0.3.tar.xz
instead of the source tarball produced by GitHub.

RabbitMQ 4.0.2

RabbitMQ 4.0.2 is a maintenance release in the 4.0.x release series.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users and those
who hold a valid commercial support license.

Please refer to the upgrade section from the 4.0 release notes
if upgrading from a version prior to 4.0.

This release requires Erlang 26 and supports Erlang versions up to 26.2.x.
RabbitMQ and Erlang/OTP Compatibility Matrix has more details on
Erlang version requirements for RabbitMQ.

Minimum Supported Erlang Version

As of 4.0, RabbitMQ requires Erlang 26. Nodes will fail to start on older Erlang releases.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Generic Binary Package

Bug Fixes

• Generic binary packages used an incorrect version (4.0.0+2 instead of 4.0.1) at build time

  GitHub issue: #12339

Dependency Changes

None in this release.

Source Code Archives

To obtain source code of the entire distribution, please download the archive named rabbitmq-server-4.0.2.tar.xz
instead of the source tarball produced by GitHub.

RabbitMQ 4.0.1

RabbitMQ 4.0 is a new major release.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users
and those who hold a valid commercial support license.

Highlights

Some key improvements in this release are listed below.

• Khepri, an alternative schema data store developed to replace Mnesia,
  has matured and is now fully supported (it previously was an experimental feature)
• AMQP 1.0 is now a core protocol that is always enabled. Its plugin is now a no-op that only exists to simplify upgrades.
• The AMQP 1.0 implementation is now significantly more efficient: its peak throughput is more than double than that of 3.13.x
  on some workloads
• Efficient sub-linear quorum queue recovery on node startup using checkpoints
• Quorum queues now support priorities (but not exactly the same way as classic queues)
• AMQP 1.0 clients now can manage topologies similarly to how AMQP 0-9-1 clients do it
• The AMQP 1.0 convention (address format) used for interacting with with AMQP 0-9-1 entities is now easier to reason about
• Mirroring (replication) of classic queues was removed after several years of deprecation. For replicated messaging data types,
  use quorum queues and/or streams. Non-replicated classic queues remain and their development continues
• Classic queue storage efficiency improvements, in particular recovery time and storage of multi-MiB messages
• Nodes with multiple enabled plugins and little on disk data to recover now start up to 20-30% faster
• New exchange type: Local Random Exchange

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

Breaking Changes and Compatibility Notes

Classic Queues is Now a Non-Replicated Queue Type

After three years of deprecation, classic queue mirroring was completely removed in this version.
Quorum queues and streams are two mature
replicated data types offered by RabbitMQ 4.x. Classic queues continue being supported without any breaking changes
for client libraries and applications but they are now a non-replicated queue type.

After an upgrade to 4.0, all classic queue mirroring-related parts of policies will have no effect.
Classic queues will continue to work like before but with only one replica.

Clients will be able to connect to any node to publish to and consume from any non-replicated classic queues.
Therefore applications will be able to use the same classic queues as before.

See Mirrored Classic Queues Migration to Quorum Queues for guidance
on how to migrate to quorum queues for the parts of the system that really need to use replication.

Quorum Queues Now Have a Default Redelivery Limit

Quorum queues now have a default redelivery limit set to 20.
Messages that are redelivered 20 times or more will be dead-lettered or dropped (removed).

This limit is necessary to protect nodes from consumers that run into infinite fail-requeue-fail-requeue loops. Such
consumers can drive a node out of disk space by making a quorum queue Raft log grow forever without allowing compaction
of older entries to happen.

If 20 deliveries per message is a common scenario for a queue, a dead-lettering target or a higher limit must be configured
for such queues. The recommended way of doing that is via a policy.
See the Position Messaging Handling section
in the quorum queue documentation guide.

Note that increasing the limit is recommended against: usually the presence of messages that have been redelivered 20 times or more suggests
that a consumer has entered a fail-requeue-fail-requeue loop, in which case even a much higher limit
won't help avoid the dead-lettering.

For specific cases where the RabbitMQ configuration cannot be updated to include a dead letter policy
the delivery limit can be disabled by setting a delivery limit configuration of -1. However, the RabbitMQ team
strongly recommends keeping the delivery limit in place to ensure cluster availability isn't
accidentally sacrificed.

AMQP 0.9.1 x-death header

Up to RabbitMQ 3.13, when an AMQP 0.9.1 client (re-)published a message to RabbitMQ, RabbitMQ interpreted the
AMQP 0.9.1 x-death header in the published message's basic_message.content.properties.headers field.

RabbitMQ 4.x will not interpret this x-death header anymore when clients (re-)publish a message.
Note that RabbitMQ 4.x will continue to set and update the x-death header every time a message is dead-lettered, including when a client rejects the message.

Applications that rely on RabbitMQ incrementing the count fields within the x-death header array elements for new messages (re-)published
(instead of existing messages being rejected), should introduce and increment a separate x- header,
with a name that would not be updated by RabbitMQ itself.

CQv1 Storage Implementation was Removed

CQv1, the original classic queue storage layer, was removed
except for the part that's necessary for upgrades to CQv2 (the 2nd generation).

In case rabbitmq.conf explicitly sets classic_queue.default_version to 1 like so

# this configuration value is no longer supported,
# remove this line or set the version to 2
classic_queue.default_version = 1

nodes will now fail to start. Removing the line will make the node start and perform
the migration from CQv1 to CQv2.

Settings cluster_formation.randomized_startup_delay_range.* were Removed

The following two deprecated rabbitmq.conf settings were removed:

cluster_formation.randomized_startup_delay_range.min
cluster_formation.randomized_startup_delay_range.max

RabbitMQ 4.0 will fail to boot if these settings are configured in rabbitmq.conf.

Several Disk I/O-Related Metrics were Removed

Several I/O-related metrics are dropped, they should be monitored at the infrastructure and kernel layers

Default Maximum Message Size Reduced to 16 MiB

Default maximum message size is reduced to 16 MiB (from 128 MiB).

The limit can be increased via a rabbitmq.conf setting:

# 32 MiB
max_message_size = 33554432

However, it is recommended that such large multi-MiB messages are put into a blob store, and their
IDs are passed around in messages instead of the entire payload.

AMQP 1.0

RabbitMQ 3.13 rabbitmq.conf setting rabbitmq_amqp1_0.default_vhost is unsupported in RabbitMQ 4.0.

Instead default_vhost will be used to determine the default vhost an AMQP 1.0 client connects to(i.e. when the AMQP 1.0 client
does not define the vhost in the hostname field of the open frame).

Starting with RabbitMQ 4.0, RabbitMQ strictly validates that
delivery annotations,
message annotations, and
footer contain only
non-reserved annotation keys.
As a result, clients can only send symbolic keys that begin with x-.

MQTT

RabbitMQ 3.13 rabbitmq.conf settings mqtt.default_user, mqtt.default_password,
and amqp1_0.default_user are unsupported in RabbitMQ 4.0.

Instead, set the new RabbitMQ 4.0 settings anonymous_login_user and anonymous_login_pass (both values default to guest).
For production scenarios, disallow anonymous logins.

TLS Client (LDAP, Shovels, Federation) Defaults

Starting with Erlang 26, client side TLS peer certificate chain verification settings are enabled by default in most contexts:
from federation links to shovels to TLS-enabled LDAP client connections.

If using TLS peer certificate chain verification is not practical or necessary, it can be disabled.
Please refer to the docs of the feature in question, for example,
this one on TLS-enabled LDAP client connections,
two others on [TLS-enabl...

RabbitMQ 4.0.0-rc.2

RabbitMQ 4.0.0-rc.2 is a candidate of a new major release.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users
and those who hold a valid commercial support license.

Highlights

Some key improvements in this release are listed below.

• Khepri, an alternative schema data store developed to replace Mnesia,
  has matured and is now fully supported (it previously was an experimental feature)
• AMQP 1.0 is now a core protocol that is always enabled. Its plugin is now a no-op that only exists to simplify upgrades.
• The AMQP 1.0 implementation is now significantly more efficient: its peak throughput is more than double than that of 3.13.x
  on some workloads
• Efficient sub-linear quorum queue recovery on node startup using checkpoints
• Quorum queues now support priorities (but not exactly the same way as classic queues)
• AMQP 1.0 clients now can manage topologies similarly to how AMQP 0-9-1 clients do it
• The AMQP 1.0 convention (address format) used for interacting with with AMQP 0-9-1 entities is now easier to reason about
• Mirroring (replication) of classic queues was removed after several years of deprecation. For replicated messaging data types,
  use quorum queues and/or streams. Non-replicated classic queues remain and their development continues
• Classic queue storage efficiency improvements, in particular recovery time and storage of multi-MiB messages
• Nodes with multiple enabled plugins and little on disk data to recover now start up to 20-30% faster
• New exchange type: Local Random Exchange

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

Breaking Changes and Compatibility Notes

Classic Queues is Now a Non-Replicated Queue Type

After three years of deprecated, classic queue mirroring was completely removed in this version.
Quorum queues and streams are two mature
replicated data types offered by RabbitMQ 4.x. Classic queues continue being supported without any breaking changes
for client libraries and applications but they are now a non-replicated queue type.

After an upgrade to 4.0, all classic queue mirroring-related parts of policies will have no effect.
Classic queues will continue to work like before but with only one replica.

Clients will be able to connect to any node to publish to and consume from any non-replicated classic queues.
Therefore applications will be able to use the same classic queues as before.

See Mirrored Classic Queues Migration to Quorum Queues for guidance
on how to migrate to quorum queues for the parts of the system that really need to use replication.

Quorum Queues Now Have a Default Redelivery Limit

Quorum queues now have a default redelivery limit set to 20.
Messages that are redelivered 20 times or more will be dead-lettered or dropped (removed).

This limit is necessary to protect nodes from consumers that run into infinite fail-requeue-fail-requeue loops. Such
consumers can drive a node out of disk space by making a quorum queue Raft log grow forever without allowing compaction
of older entries to happen.

If 20 deliveries per message is a common scenario for a queue, a dead-lettering target or a higher limit must be configured
for such queues. The recommended way of doing that is via a policy.
See the Position Messaging Handling section
in the quorum queue documentation guide.

Note that increasing the limit is recommended against: usually the presence of messages that have been redelivered 20 times or more suggests
that a consumer has entered a fail-requeue-fail-requeue loop, in which case even a much higher limit
won't help avoid the dead-lettering.

For specific cases where the RabbitMQ configuration cannot be updated to include a dead letter policy
the delivery limit can be disabled by setting a delivery limit configuration of -1. However, the RabbitMQ team
strongly recommends keeping the delivery limit in place to ensure cluster availability isn't
accidentally sacrificed.

CQv1 Storage Implementation was Removed

CQv1, the original classic queue storage layer, was removed
except for the part that's necessary for upgrades to CQv2 (the 2nd generation).

In case rabbitmq.conf explicitly sets classic_queue.default_version to 1 like so

# this configuration value is no longer supported,
# remove this line or set the version to 2
classic_queue.default_version = 1

nodes will now fail to start. Removing the line will make the node start and perform
the migration from CQv1 to CQv2.

Settings cluster_formation.randomized_startup_delay_range.* were Removed

The following two deprecated rabbitmq.conf settings were removed:

cluster_formation.randomized_startup_delay_range.min
cluster_formation.randomized_startup_delay_range.max

RabbitMQ 4.0 will fail to boot if these settings are configured in rabbitmq.conf.

Several Disk I/O-Related Metrics were Removed

Several I/O-related metrics are dropped, they should be monitored at the infrastructure and kernel layers

Default Maximum Message Size Reduced to 16 MiB

Default maximum message size is reduced to 16 MiB (from 128 MiB).

The limit can be increased via a rabbitmq.conf setting:

# 32 MiB
max_message_size = 33554432

However, it is recommended that such large multi-MiB messages are put into a blob store, and their
IDs are passed around in messages instead of the entire payload.

AMQP 1.0

RabbitMQ 3.13 rabbitmq.conf setting rabbitmq_amqp1_0.default_vhost is unsupported in RabbitMQ 4.0.

Instead default_vhost will be used to determine the default vhost an AMQP 1.0 client connects to(i.e. when the AMQP 1.0 client
does not define the vhost in the hostname field of the open frame).

MQTT

RabbitMQ 3.13 rabbitmq.conf settings mqtt.default_user, mqtt.default_password,
and amqp1_0.default_user are unsupported in RabbitMQ 4.0.

Instead, set the new RabbitMQ 4.0 settings anonymous_login_user and anonymous_login_pass (both values default to guest).
For production scenarios, disallow anonymous logins.

TLS Client (LDAP, Shovels, Federation) Defaults

Starting with Erlang 26, client side TLS peer certificate chain verification settings are enabled by default in most contexts:
from federation links to shovels to TLS-enabled LDAP client connections.

If using TLS peer certificate chain verification is not practical or necessary, it can be disabled.
Please refer to the docs of the feature in question, for example,
this one on TLS-enabled LDAP client connections,
two others on TLS-enabled dynamic shovels and dynamic shovel URI query parameters.

Shovels

RabbitMQ Shovels will be able connect to a RabbitMQ 4.0 node via AMQP 1.0 only when the Shovel runs on a RabbitMQ node >= 3.13.7.

TLS-enabled Shovels will be affected by the TLS client default changes in Erlang 26 (see above).

Erlang/OTP Compatibility Notes

This release requires Erlang 26.2.

Provisioning Latest Erlang Releases explains
what package repositories and tools can be used to provision latest patch versions of Erlang 26.x.

Release Artifacts

RabbitMQ releases are distributed via GitHub.
Debian and RPM packages are available via
repositories maintained by the RabbitMQ Core Team.

Community Docker image, Chocolatey package, and the Homebrew formula
are other installation options. They are updated with a delay.

Upgrading to 4.0

Documentation guides on upgrades

See the Upgrading guide for documentation on upgrades and GitHub releases
for rele...

RabbitMQ 4.0.0-rc.1

RabbitMQ 4.0.0-rc.1 is a candidate of a new major release.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users
and those who hold a valid commercial support license.

Highlights

Some key improvements in this release are listed below.

• Khepri, an alternative schema data store developed to replace Mnesia, has matured
• AMQP 1.0 is now a core protocol that is always enabled. Its plugin is now a no-op that only exists to simplify upgrades.
• The AMQP 1.0 implementation is now significantly more efficient: its peak throughput is more than double than that of 3.13.x
  on some workloads
• Efficient sub-linear quorum queue recovery on node startup using checkpoints
• Quorum queues now support priorities (but not exactly the same way as classic queues)
• AMQP 1.0 clients now can manage topologies similarly to how AMQP 0-9-1 clients do it
• The AMQP 1.0 convention (address format) used for interacting with with AMQP 0-9-1 entities is now easier to reason about
• Mirroring (replication) of classic queues was removed after several years of deprecation. For replicated messaging data types,
  use quorum queues and/or streams. Non-replicated classic queues remain and their development continues
• Classic queue storage efficiency improvements, in particular recovery time and storage of multi-MiB messages
• Nodes with multiple enabled plugins and little on disk data to recover now start up to 20-30% faster
• New exchange type: Local Random Exchange

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

Breaking Changes and Compatibility Notes

Classic Queues is Now a Non-Replicated Queue Type

After three years of deprecated, classic queue mirroring was completely removed in this version.
Quorum queues and streams are two mature
replicated data types offered by RabbitMQ 4.x. Classic queues continue being supported without any breaking changes
for client libraries and applications but they are now a non-replicated queue type.

After an upgrade to 4.0, all classic queue mirroring-related parts of policies will have no effect.
Classic queues will continue to work like before but with only one replica.

Clients will be able to connect to any node to publish to and consume from any non-replicated classic queues.
Therefore applications will be able to use the same classic queues as before.

See Mirrored Classic Queues Migration to Quorum Queues for guidance
on how to migrate to quorum queues for the parts of the system that really need to use replication.

Quorum Queues Now Have a Default Redelivery Limit

Quorum queues now have a default redelivery limit set to 20.
Messages that are redelivered 20 times or more will be dead-lettered or dropped (removed).

This limit is necessary to protect nodes from consumers that run into infinite fail-requeue-fail-requeue loops. Such
consumers can drive a node out of disk space by making a quorum queue Raft log grow forever without allowing compaction
of older entries to happen.

If 20 deliveries per message is a common scenario for a queue, a dead-lettering target or a higher limit must be configured
for such queues. The recommended way of doing that is via a policy.
See the Position Messaging Handling section
in the quorum queue documentation guide.

Note that increasing the limit is recommended against: usually the presence of messages that have been redelivered 20 times or more suggests
that a consumer has entered a fail-requeue-fail-requeue loop, in which case even a much higher limit
won't help avoid the dead-lettering.

For specific cases where the RabbitMQ configuration cannot be updated to include a dead letter policy
the delivery limit can be disabled by setting a delivery limit configuration of -1. However, the RabbitMQ team
strongly recommends keeping the delivery limit in place to ensure cluster availability isn't
accidentally sacrificed.

CQv1 Storage Implementation was Removed

CQv1, the original classic queue storage layer, was removed
except for the part that's necessary for upgrades to CQv2 (the 2nd generation).

In case rabbitmq.conf explicitly sets classic_queue.default_version to 1 like so

# this configuration value is no longer supported,
# remove this line or set the version to 2
classic_queue.default_version = 1

nodes will now fail to start. Removing the line will make the node start and perform
the migration from CQv1 to CQv2.

Settings cluster_formation.randomized_startup_delay_range.* were Removed

The following two deprecated rabbitmq.conf settings were removed:

cluster_formation.randomized_startup_delay_range.min
cluster_formation.randomized_startup_delay_range.max

RabbitMQ 4.0 will fail to boot if these settings are configured in rabbitmq.conf.

Several Disk I/O-Related Metrics were Removed

Several I/O-related metrics are dropped, they should be monitored at the infrastructure and kernel layers

Default Maximum Message Size Reduced to 16 MiB

Default maximum message size is reduced to 16 MiB (from 128 MiB).

The limit can be increased via a rabbitmq.conf setting:

# 32 MiB
max_message_size = 33554432

However, it is recommended that such large multi-MiB messages are put into a blob store, and their
IDs are passed around in messages instead of the entire payload.

AMQP 1.0

RabbitMQ 3.13 rabbitmq.conf setting rabbitmq_amqp1_0.default_vhost is unsupported in RabbitMQ 4.0.

Instead default_vhost will be used to determine the default vhost an AMQP 1.0 client connects to(i.e. when the AMQP 1.0 client
does not define the vhost in the hostname field of the open frame).

MQTT

RabbitMQ 3.13 rabbitmq.conf settings mqtt.default_user, mqtt.default_password,
and amqp1_0.default_user are unsupported in RabbitMQ 4.0.

Instead, set the new RabbitMQ 4.0 settings anonymous_login_user and anonymous_login_pass (both values default to guest).
For production scenarios, disallow anonymous logins.

Shovels

RabbitMQ Shovels will be able connect to a RabbitMQ 4.0 node via AMQP 1.0 only when the Shovel runs on a RabbitMQ node >= 3.13.7.

Erlang/OTP Compatibility Notes

This release requires Erlang 26.2.

Provisioning Latest Erlang Releases explains
what package repositories and tools can be used to provision latest patch versions of Erlang 26.x.

Release Artifacts

RabbitMQ releases are distributed via GitHub.
Debian and RPM packages are available via
repositories maintained by the RabbitMQ Core Team.

Community Docker image, Chocolatey package, and the Homebrew formula
are other installation options. They are updated with a delay.

Upgrading to 4.0

Documentation guides on upgrades

See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.

This release series only supports upgrades from 3.13.x.

This release requires all feature flags in the 3.x series (specifically 3.13.x) to be enabled before upgrading,
there is no upgrade path from 3.12.14 (or a later patch release) straight to 4.0.0.

Required Feature Flags

This release graduates all feature flags introduced up to 3.13.0.

All users must enable all stable [feature flags] before upgrading to 4.0 from
the latest available 3.13.x patch release.

Mixed version cluster compatibility

RabbitMQ 4.0.0 nodes can run alongside 3.13.x nodes. 4.0.x-specific features can only be made available when all nodes in the cluster
upgrade to 4.0.0 or a later patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected. T...

RabbitMQ 4.0.0-beta.6

RabbitMQ 4.0.0-beta.6 is a preview of a new major release.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users and those who hold a valid commercial support license.

Highlights

Some key improvements in this release are listed below.

• Khepri, an alternative schema data store developed to replace Mnesia, has matured
• AMQP 1.0 is now a core protocol that is always enabled. Its plugin is now a no-op that only exists to simplify upgrades.
• The AMQP 1.0 implementation is now significantly more efficient: its peak throughput is more than double than that of 3.13.x
  on some workloads
• Efficient sub-linear quorum queue recovery on node startup using checkpoints
• Quorum queues now support priorities (but not exactly the same way as classic queues)
• AMQP 1.0 clients now can manage topologies similarly to how AMQP 0-9-1 clients do it
• The AMQP 1.0 convention (address format) used for interacting with with AMQP 0-9-1 entities is now easier to reason about
• Mirroring (replication) of classic queues was removed after several years of deprecation. For replicated messaging data types,
  use quorum queues and/or streams. Non-replicated classic queues remain and their development continues
• Classic queue storage efficiency improvements, in particular recovery time and storage of multi-MiB messages
• Nodes with multiple enabled plugins and little on disk data to recover now start up to 20-30% faster
• New exchange type: Local Random Exchange

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

Breaking Changes and Compatibility Notes

Classic Queues is Now a Non-Replicated Queue Type

After three years of deprecated, classic queue mirroring was completely removed in this version.
Quorum queues and streams are two mature
replicated data types offered by RabbitMQ 4.x. Classic queues continue being supported without any breaking changes
for client libraries and applications but they are now a non-replicated queue type.

After an upgrade to 4.0, all classic queue mirroring-related parts of policies will have no effect.
Classic queues will continue to work like before but with only one replica.

Clients will be able to connect to any node to publish to and consume from any non-replicated classic queues.
Therefore applications will be able to use the same classic queues as before.

See Mirrored Classic Queues Migration to Quorum Queues for guidance
on how to migrate to quorum queues for the parts of the system that really need to use replication.

Quorum Queues Now Have a Default Redelivery Limit

Quorum queues now have a default redelivery limit set to 20.

CQv1 Storage Implementation was Removed

CQv1, the original classic queue storage layer, was removed
except for the part that's necessary for upgrades to CQv2 (the 2nd generation).

In case rabbitmq.conf explicitly sets classic_queue.default_version to 1 like so

# this configuration value is no longer supported,
# remove this line or set the version to 2
classic_queue.default_version = 1

nodes will now fail to start. Removing the line will make the node start and perform
the migration from CQv1 to CQv2.

Settings cluster_formation.randomized_startup_delay_range.* were Removed

The following two deprecated rabbitmq.conf settings were removed:

cluster_formation.randomized_startup_delay_range.min
cluster_formation.randomized_startup_delay_range.max

RabbitMQ 4.0 will fail to boot if these settings are configured in rabbitmq.conf.

Several Disk I/O-Related Metrics were Removed

Several I/O-related metrics are dropped, they should be monitored at the infrastructure and kernel layers

Default Maximum Message Size Reduced to 16 MiB

Default maximum message size is reduced to 16 MiB (from 128 MiB).

The limit can be increased via a rabbitmq.conf setting:

# 32 MiB
max_message_size = 33554432

However, it is recommended that such large multi-MiB messages are put into a blob store, and their
IDs are passed around in messages instead of the entire payload.

AMQP 1.0

RabbitMQ 3.13 rabbitmq.conf setting rabbitmq_amqp1_0.default_vhost is unsupported in RabbitMQ 4.0.

Instead default_vhost will be used to determine the default vhost an AMQP 1.0 client connects to(i.e. when the AMQP 1.0 client does not define the vhost in the hostname field of the open frame).

MQTT

RabbitMQ 3.13 rabbitmq.conf settings mqtt.default_user, mqtt.default_password,
and amqp1_0.default_user are unsupported in RabbitMQ 4.0.

Instead, set the new RabbitMQ 4.0 settings anonymous_login_user and anonymous_login_pass (both values default to guest).
For production scenarios, disallow anonymous logins.

Shovels

RabbitMQ Shovels will be able connect to a RabbitMQ 4.0 node via AMQP 1.0 only when the Shovel runs on a RabbitMQ node >= 3.13.7.

Erlang/OTP Compatibility Notes

This release requires Erlang 26.2.

Provisioning Latest Erlang Releases explains
what package repositories and tools can be used to provision latest patch versions of Erlang 26.x.

Release Artifacts

RabbitMQ releases are distributed via GitHub.
Debian and RPM packages are available via
repositories maintained by the RabbitMQ Core Team.

Community Docker image, Chocolatey package, and the Homebrew formula
are other installation options. They are updated with a delay.

Upgrading to 4.0

Documentation guides on upgrades

See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.

This release series only supports upgrades from 3.13.x.

This release requires all feature flags in the 3.x series (specifically 3.13.x) to be enabled before upgrading,
there is no upgrade path from 3.12.14 (or a later patch release) straight to 4.0.0.

Required Feature Flags

This release graduates all feature flags introduced up to 3.13.0.

All users must enable all stable [feature flags] before upgrading to 4.0 from
the latest available 3.13.x patch release.

Mixed version cluster compatibility

RabbitMQ 4.0.0 nodes can run alongside 3.13.x nodes. 4.0.x-specific features can only be made available when all nodes in the cluster
upgrade to 4.0.0 or a later patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes will be covered in future updates.
Once all nodes are upgraded to 4.0.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).

Recommended Post-upgrade Procedures

Set a low priority dead lettering policy for all quorum queues to dead letter to a stream or similar
so that messages that reach the new default delivery limit of 20 aren't lost completely
when no dead lettering policy is in place.

TBD

Changes Worth Mentioning

This section is incomplete and will be expanded as 4.0 approaches its release candidate stage.

Core Server

Enhancements

• Efficient sub-linear quorum queue recovery on node startup using checkpoints.

  GitHub issue: #10637

• Classic queue storage v2 (CQv2) optimizations. For example, CQv2 recovery time on node boot
  is now twice as fast for some data sets.

  GitHub issue: #11112

• Node startup time improvements. For some environments, nodes with very small on disk data sets
  now start about 25% quicker.

  GitHub issue: #10989

• Quorum queues now support priorities. However,
  there are...

RabbitMQ 3.13.7

RabbitMQ 3.13.7 is a maintenance release in the 3.13.x release series.

This upgrade is highly recommended to all users currently on earlier 3.13.x series and
in particular between 3.13.3 and 3.13.5, inclusive.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users and those
who hold a valid commercial support license.

Please refer to the upgrade section from the 3.13.0 release notes
if upgrading from a version prior to 3.13.0.

This release requires Erlang 26 and supports Erlang versions up to 26.2.x.
RabbitMQ and Erlang/OTP Compatibility Matrix has more details on
Erlang version requirements for RabbitMQ.

Minimum Supported Erlang Version

As of 3.13.0, RabbitMQ requires Erlang 26. Nodes will fail to start on older Erlang releases.

Users upgrading from 3.12.x (or older releases) on Erlang 25 to 3.13.x on Erlang 26
(both RabbitMQ and Erlang are upgraded at the same time) must consult
the v3.12.0 release notes and v3.13.0 release notes first.

Changes Worth Mentioning

Release notes can be found on GitHub at rabbitmq-server/release-notes.

Core Broker

Bug Fixes

• Streams recover better from certain node process failures that may leave behind orphaned segment files
  (that is, segment files that do not have a corresponding index file) or index files without a corresponding
  segment file.

  Kudos to @sysupbda for providing detailed reproduction steps and verifying the fix in the affected environment.

  GitHub issue: #12073

• Config file peer discovery now logs warnings for certain common user mistakes.

  GitHub issues: #11586, #11898

• Queue declaration operations now return more useful errors when Khepri is enabled and there's only a minority
  of nodes online.

  GitHub issues: #12020, #11991

• Logging is now more defensive around exception handling. Previously a (very rare) logger exception could
  lead to the amq.rabbitmq.log handler and exchange to be removed.

  Contributed by @gomoripeti.

  GitHub issue: #12107

• rabbitmq-upgrade revive unintentionally tried to perform operations on replicas that are not local to the node.
  This could result in an exceptions some of which were not handled and the command failed.
  Re-running the command usually helped.

  GitHub issue: #12038

Enhancements

• Enabling an experimental feature flag now involves an explicit confirmation.

  GitHub issue: #12059

• Khepri projections are registered in a safer manner during node boot.

  GitHub issue: #11837

MQTT

Bug Fixes

• Clients that use JWT tokens are now disconnected when their token expires. Previously all newly attempted
  operations with an expired token would be rejected but a completely passive connection was not closed.

  GitHub issue: #11869

Enhancements

• Connection that provide incorrect credentials now closed with a delay, just like for several
  other protocols supported by RabbitMQ, as a throttling mechanism.

  GitHub issue: #11906

CLI Tools

Bug Fixes

• When the Khepri feature flag is not enabled, rabbitmq-diagnostics metadata_store_status will not try to retrieve
  and display its status.

  GitHub issue: #12103

Enhancements

• rabbitmq-upgrade await_quorum_plus_one now produces more log messages when the operation times out.
  When Khepri is enabled, it now also treats Khepri as a critical Raft-based component that may depend on replica quorum
  just like queues and streams do.

  GitHub issue: #12117

Management Plugin

Bug Fixes

• When no virtual host limits are set, the limits collection was returned as a JSON array (and not a JSON object)
  by GET /api/vhost-limits.

  GitHub issue: #12084

Enhancements

• GET /api/queues/quorum/{vhost}/{name}/status is a new endpoint that allows clients to retrieve several key quorum queue
  replica and Raft metrics.

  Contributed by @SimonUnge.

  GitHub issue: #12072

Shovel Plugin

Bug Fixes

• GET /api/shovels/{vhost}/{name} now correctly returns a single shovel instead of all shovels in the target
  virtual host.

  GitHub issue: #12040

Consistent Hashing Exchange Plugin

Bug Fixes

• For an exchange declared with a hash-header, publishing failed with an exception when the client (usually unintentionally)
  did not set that header.

  GitHub issue: #11808

Dependency Changes

• Osiris was upgraded to 1.8.3
• Cuttlefish was upgraded to 3.4.0
• observer_cli was upgraded to 1.7.5

Source Code Archives

To obtain source code of the entire distribution, please download the archive named rabbitmq-server-3.13.7.tar.xz
instead of the source tarball produced by GitHub.

RabbitMQ 4.0.0-beta.5

RabbitMQ 4.0.0-beta.5 is a preview of a new major release.

Starting June 1st, 2024, community support for this series will only be provided to regularly contributing users and those who hold a valid commercial support license.

Highlights

Some key improvements in this release are listed below.

• Khepri, an alternative schema data store developed to replace Mnesia, is no longer considered an experimental feature
• AMQP 1.0 is now a core protocol that is always enabled. Its plugin is now a no-op that only exists to simplify upgrades.
• The AMQP 1.0 implementation is now significantly more efficient: its peak throughput is more than double than that of 3.13.x
  on some workloads
• Efficient sub-linear quorum queue recovery on node startup using checkpoints
• Quorum queues now support priorities (but not exactly the same way as classic queues)
• AMQP 1.0 clients now can manage topologies similarly to how AMQP 0-9-1 clients do it
• The AMQP 1.0 convention (address format) used for interacting with with AMQP 0-9-1 entities is now easier to reason about
• Mirroring (replication) of classic queues was removed after several years of deprecation. For replicated messaging data types,
  use quorum queues and/or streams. Non-replicated classic queues remain and their development continues
• Classic queue storage efficiency improvements, in particular recovery time and storage of multi-MiB messages
• Nodes with multiple enabled plugins and little on disk data to recover now start up to 20-30% faster
• New exchange type: Local Random Exchange

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

Breaking Changes and Compatibility Notes

Classic Queues is Now a Non-Replicated Queue Type

After three years of deprecated, classic queue mirroring was completely removed in this version.
Quorum queues and streams are two mature
replicated data types offered by RabbitMQ 4.x. Classic queues continue being supported without any breaking changes
for client libraries and applications but they are now a non-replicated queue type.

After an upgrade to 4.0, all classic queue mirroring-related parts of policies will have no effect.
Classic queues will continue to work like before but with only one replica.

Clients will be able to connect to any node to publish to and consume from any non-replicated classic queues.
Therefore applications will be able to use the same classic queues as before.

See Mirrored Classic Queues Migration to Quorum Queues for guidance
on how to migrate to quorum queues for the parts of the system that really need to use replication.

Quorum Queues Now Have a Default Redelivery Limit

Quorum queues now have a default redelivery limit set to 20.

CQv1 Storage Implementation was Removed

CQv1, the original classic queue storage layer, was removed
except for the part that's necessary for upgrades to CQv2 (the 2nd generation).

In case rabbitmq.conf explicitly sets classic_queue.default_version to 1 like so

# this configuration value is no longer supported,
# remove this line or set the version to 2
classic_queue.default_version = 1

nodes will now fail to start. Removing the line will make the node start and perform
the migration from CQv1 to CQv2.

Several Disk I/O-Related Metrics were Removed

Several I/O-related metrics are dropped, they should be monitored at the infrastructure and kernel layers

Default Maximum Message Size Reduced to 16 MiB

Default maximum message size is reduced to 16 MiB (from 128 MiB).

The limit can be increased via a rabbitmq.conf setting:

# 32 MiB
max_message_size = 33554432

However, it is recommended that such large multi-MiB messages are put into a blob store, and their
IDs are passed around in messages instead of the entire payload.

AMQP 1.0

RabbitMQ 3.13 rabbitmq.conf setting rabbitmq_amqp1_0.default_vhost is unsupported in RabbitMQ 4.0.

Instead default_vhost will be used to determine the default vhost an AMQP 1.0 client connects to(i.e. when the AMQP 1.0 client does not define the vhost in the hostname field of the open frame).

MQTT

RabbitMQ 3.13 rabbitmq.conf settings mqtt.default_user, mqtt.default_password,
and amqp1_0.default_user are unsupported in RabbitMQ 4.0.

Instead, set the new RabbitMQ 4.0 settings anonymous_login_user and anonymous_login_pass (both values default to guest).
For production scenarios, disallow anonymous logins.

Shovels

RabbitMQ Shovels will be able connect to a RabbitMQ 4.0 node via AMQP 1.0 only when the Shovel runs on a RabbitMQ node >= 3.13.7.

Erlang/OTP Compatibility Notes

This release requires Erlang 26.2.

Provisioning Latest Erlang Releases explains
what package repositories and tools can be used to provision latest patch versions of Erlang 26.x.

Release Artifacts

RabbitMQ releases are distributed via GitHub.
Debian and RPM packages are available via Cloudsmith mirrors.

Community Docker image, Chocolatey package, and the Homebrew formula
are other installation options. They are updated with a delay.

Upgrading to 4.0

Documentation guides on upgrades

See the Upgrading guide for documentation on upgrades and GitHub releases
for release notes of individual releases.

This release series only supports upgrades from 3.13.x.

This release requires all feature flags in the 3.x series (specifically 3.13.x) to be enabled before upgrading,
there is no upgrade path from 3.12.14 (or a later patch release) straight to 4.0.0.

Required Feature Flags

This release graduates all feature flags introduced up to 3.13.0.

All users must enable all stable [feature flags] before upgrading to 4.0 from
the latest available 3.13.x patch release.

Mixed version cluster compatibility

RabbitMQ 4.0.0 nodes can run alongside 3.13.x nodes. 4.0.x-specific features can only be made available when all nodes in the cluster
upgrade to 4.0.0 or a later patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes will be covered in future updates.
Once all nodes are upgraded to 4.0.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).

Recommended Post-upgrade Procedures

Set a low priority dead lettering policy for all quorum queues to dead letter to a stream or similar
so that messages that reach the new default delivery limit of 20 aren't lost completely
when no dead lettering policy is in place.

TBD

Changes Worth Mentioning

This section is incomplete and will be expanded as 4.0 approaches its release candidate stage.

Core Server

Enhancements

• Efficient sub-linear quorum queue recovery on node startup using checkpoints.

  GitHub issue: #10637

• Classic queue storage v2 (CQv2) optimizations. For example, CQv2 recovery time on node boot
  is now twice as fast for some data sets.

  GitHub issue: #11112

• Node startup time improvements. For some environments, nodes with very small on disk data sets
  now start about 25% quicker.

  GitHub issue: #10989

• Quorum queues now support priorities. However,
  there are difference with how priorities work in classic queues.

  GitHub issue: #10637

• Per-message metadata stored in the quorum queue Raft log now uses less disk space.

  GitHub issue: #8261

• Single Active Consumer (SAC) implementation of quorum queues now respects consumer priorities....