2.15.0-rc.0
Pre-releaseThis release contains 493 PRs from 61 authors, including new contributors Alexander Akhmetov, Daan Schipper, Daniel Kovacs, I. Elisa Pasaoglu, Jay Clifford, Jorge Alberto Díaz Orozco (Akiel), Martin Valiente Ainz, Mia, Michael Tweten, Nikos Angelopoulos, Santi Leira, cui fliter, elsoa-invitech, madhu-reddy-peram. Thank you!
Grafana Mimir version 2.15.0-rc.0 release notes
Grafana Labs is excited to announce version 2.15 of Grafana Mimir.
The highlights that follow include the top features, enhancements, and bug fixes in this release.
For the complete list of changes, refer to the CHANGELOG.
Features and enhancements
S2 compression for gRPC is now supported using the following flags:
-alertmanager.alertmanager-client.grpc-compression=s2
-ingester.client.grpc-compression=s2
-querier.frontend-client.grpc-compression=s2
-querier.scheduler-client.grpc-compression=s2
-query-frontend.grpc-client-config.grpc-compression=s2
-query-scheduler.grpc-client-config.grpc-compression=s2
-ruler.client.grpc-compression=s2
-ruler.query-frontend.grpc-client-config.grpc-compression=s2
Distributors now support lz4
OTLP compression, and can be deployed in multiple availability zones.
The ruler's <prometheus-http-prefix>/api/v1/rules
endpoint now supports exclude_alerts
, group_limit
, and group_next_token
parameters.
mimirtool's analyze ruler/prometheus commands now support bearer tokens.
HTTP client settings can now be tuned for GCS and Azure backends via an http
block or corresponding CLI flags.
The compactor now refreshes deletion marks concurrently when updating the bucket index.
The number of Memcached replicas for each type of cache can now be set using configuration settings when using jsonnet.
Important changes
In Grafana Mimir 2.15, the following behavior has changed:
The following alertmanager metrics are not exported for a user
when the metric value is zero:
cortex_alertmanager_alerts_received_total
cortex_alertmanager_alerts_invalid_total
cortex_alertmanager_partial_state_merges_total
cortex_alertmanager_partial_state_merges_failed_total
cortex_alertmanager_state_replication_total
cortex_alertmanager_state_replication_failed_total
cortex_alertmanager_alerts
cortex_alertmanager_silences
PromQL compatibility has been upgraded from Prometheus 2.0 to 3.0. More details can be found in the Prometheus documentation. The following changes are of note:
- The
.
pattern in regular expressions in PromQL now matches newline characters. - Lookback and range selectors are left open and right closed (previously left closed and right closed).
- Native histograms now use exponential interpolation.
Backwards compatibility in dashboards and alerts for thanos_memcached_
-prefixed metrics has been removed. These metrics were removed in 2.12 in favor of thanos_cache_
-prefixed metrics.
Experimental support for Redis as a cache backend has been removed from jsonnet.
The following deprecated configuration options were removed in this release:
-distributor.direct-otlp-translation-enabled
, which has been enabled by default since 2.13 and is now considered stable.-query-scheduler.prioritize-query-components
, which is always enabled now-ingest-storage.kafka.ongoing-records-per-fetch
and-ingest-storage.kafka.startup-records-per-fetch
, which have been removed in favour of-ingest-storage.kafka.max-buffered-bytes
-api.get-request-for-ingester-shutdown-enabled
, a deprecated experimental flag which has been marked for removal in 2.15.
Experimental features
Grafana Mimir 2.15 includes some features that are experimental and disabled by default.
Use these features with caution and report any issues that you encounter:
Mimir's experimental PromQL engine can now be enabled with -querier.query-engine=mimir
. This new engine provides improved performance and reduced querier resource consumption; however, it supports only a subset of all PromQL features. It will fall back to Prometheus' engine for queries containing unsupported features.
The query-frontend can now cache non-transient errors using the experimental flags -query-frontend.cache-errors
and -query-frontend.results-cache-ttl-for-errors
.
The query-frontend and querier both support an experimental PromQL function, double_exponential_smoothing
, which can be enabled by setting -querier.promql-experimental-functions-enabled=true
and -query-frontend.promql-experimental-functions-enabled=true
.
The ingester can now support out-of-order native histogram ingestion via the flag -ingester.ooo-native-histograms-ingestion-enabled
.
The ingester can now build 24h blocks for out-of-order data which is >24h old, using the setting -blocks-storage.tsdb.bigger-out-of-order-blocks-for-old-samples
.
The ruler now supports caching the contents of rule groups via the setting -ruler-storage.cache.rule-group-enabled
.
The distributor now supports promotion of OTel resource attributes to labels via the setting -distributor.promote-otel-resource-attributes
.
Bug fixes
- Alerts: Fix autoscaling metrics joins in
MimirAutoscalerNotActive
when series churn. - Alerts: Exclude failed cache "add" operations from alerting since failures are expected in normal operation.
- Alerts: Exclude read-only replicas from
IngesterInstanceHasNoTenants
alert. - Alerts: Use resident set memory for the
EtcdAllocatingTooMuchMemory
alert so that ephemeral file cache memory doesn't cause the alert to misfire. - Dashboards: Fix autoscaling metrics joins when series churn.
- Distributor: Fix pooling buffer reuse logic when
-distributor.max-request-pool-buffer-size
is set. - Ingester: Fix issue where active series requests error when encountering a stale posting.
- Ingester: Fix race condition in per-tenant TSDB creation.
- Ingester: Fix race condition in exemplar adding.
- Ingester: Fix race condition in native histogram appending.
- Ingester: Fix bug in concurrent fetching where a failure to list topics on startup would cause to use an invalid topic ID (0x00000000000000000000000000000000).
- Ingester: Fix data loss bug in the experimental ingest storage when a Kafka Fetch is split into multiple requests and some of them return an error.
- Ingester: Fix bug where chunks could have one unnecessary zero byte at the end.
- OTLP: Support integer exemplar value type.
- OTLP receiver: Preserve colons and combine multiple consecutive underscores into one when generating metric names in suffix adding mode (
-distributor.otel-metric-suffixes-enabled
). - Prometheus: Fix issue where negation of native histograms (e.g.
-some_native_histogram_series
) did nothing. - Prometheus: Always return unknown hint for first sample in non-gauge native histograms chunk to avoid incorrect counter reset hints when merging chunks from different sources.
- Prometheus: Ensure native histograms counter reset hints are corrected when merging results from different sources.
- PromQL: Fix issue where functions such as
rate()
over native histograms could return incorrect values if a float stale marker was present in the selected range. - PromQL:
round
now removes the metric name again. - PromQL: Fix issue where
metric might not be a counter, name does not end in _total/_sum/_count/_bucket
annotation would be emitted even ifrate
orincrease
did not have enough samples to compute a result. - Querier: Fix the behaviour of binary operators between native histograms and floats.
- Querier: Fix stddev+stdvar aggregations to always ignore native histograms, and to treat Infinity consistently.
- Query-frontend: Fix issue where sharded queries could return annotations with incorrect or confusing position information.
- Query-frontend: Fix issue where downstream consumers may not generate correct cache keys for experimental error caching.
- Query-frontend: Support
X-Read-Consistency-Offsets
on labels queries too. - Ruler: Fix issue when using the experimental
-ruler.max-independent-rule-evaluation-concurrency
feature, where the ruler could panic as it updates a running ruleset or shutdowns.
Helm chart improvements
The Grafana Mimir and Grafana Enterprise Metrics Helm charts are released independently.
Refer to the Grafana Mimir Helm chart documentation.
Changelog
2.15.0-rc.0
Grafana Mimir
- [CHANGE] Alertmanager: the following metrics are not exported for a given
user
when the metric value is zero: #9359cortex_alertmanager_alerts_received_total
cortex_alertmanager_alerts_invalid_total
cortex_alertmanager_partial_state_merges_total
cortex_alertmanager_partial_state_merges_failed_total
cortex_alertmanager_state_replication_total
cortex_alertmanager_state_replication_failed_total
cortex_alertmanager_alerts
cortex_alertmanager_silences
- [CHANGE] Distributor: Drop experimental
-distributor.direct-otlp-translation-enabled
flag, since direct OTLP translation is well tested at this point. #9647 - [CHANGE] Ingester: Change
-initial-delay
for circuit breakers to begin when the first request is received, rather than at breaker activation. #9842 - [CHANGE] Query-frontend: apply query pruning before query sharding instead of after. #9913
- [CHANGE] Ingester: remove experimental flags
-ingest-storage.kafka.ongoing-records-per-fetch
and-ingest-storage.kafka.startup-records-per-fetch
. They are removed in favour of-ingest-storage.kafka.max-buffered-bytes
. #9906 - [CHANGE] Ingester: Replace
cortex_discarded_samples_total
label fromsample-out-of-bounds
tosample-timestamp-too-old
. #9885 - [CHANGE] Ruler: the
/prometheus/config/v1/rules
does not return an error anymore if a rule group is missing in the object storage after been successfully returned by listing the storage, because it could have been deleted in the meanwhile. #9936 - [CHANGE] Querier: The
.
pattern in regular expressions in PromQL matches newline characters. With this change regular expressions like.*
match strings that include\n
. To maintain the old behaviour, you will have to change regular expressions by replacing all.
patterns with[^\n]
, e.g.foo[^\n]*
. This upgrades PromQL compatibility from Prometheus 2.0 to 3.0. #9844 - [CHANGE] Querier: Lookback and range selectors are left open and right closed (previously left closed and right closed). This change affects queries when the evaluation time perfectly aligns with the sample timestamps. For example assume querying a timeseries with evenly spaced samples exactly 1 minute apart. Previously, a range query with
5m
would usually return 5 samples, or 6 samples if the query evaluation aligns perfectly with a scrape. Now, queries like this will always return 5 samples. This upgrades PromQL compatibility from Prometheus 2.0 to 3.0. #9844 - [CHANGE] Querier: promql(native histograms): Introduce exponential interpolation. #9844
- [CHANGE] Remove deprecated
api.get-request-for-ingester-shutdown-enabled
setting, which scheduled for removal in 2.15. #10197 - [FEATURE] Querier: add experimental streaming PromQL engine, enabled with
-querier.query-engine=mimir
. #10067 - [FEATURE] Distributor: Add support for
lz4
OTLP compression. #9763 - [FEATURE] Query-frontend: added experimental configuration options
query-frontend.cache-errors
andquery-frontend.results-cache-ttl-for-errors
to allow non-transient responses to be cached. When set totrue
error responses from hitting limits or bad data are cached for a short TTL. #9028 - [FEATURE] Query-frontend: add middleware to control access to specific PromQL experimental functions on a per-tenant basis. #9798
- [FEATURE] gRPC: Support S2 compression. #9322
-alertmanager.alertmanager-client.grpc-compression=s2
-ingester.client.grpc-compression=s2
-querier.frontend-client.grpc-compression=s2
-querier.scheduler-client.grpc-compression=s2
-query-frontend.grpc-client-config.grpc-compression=s2
-query-scheduler.grpc-client-config.grpc-compression=s2
-ruler.client.grpc-compression=s2
-ruler.query-frontend.grpc-client-config.grpc-compression=s2
- [FEATURE] Alertmanager: limit added for maximum size of the Grafana state (
-alertmanager.max-grafana-state-size-bytes
). #9475 - [FEATURE] Alertmanager: limit added for maximum size of the Grafana configuration (
-alertmanager.max-config-size-bytes
). #9402 - [FEATURE] Ingester: Experimental support for ingesting out-of-order native histograms. This is disabled by default and can be enabled by setting
-ingester.ooo-native-histograms-ingestion-enabled
totrue
. #7175 - [FEATURE] Distributor: Added
-api.skip-label-count-validation-header-enabled
option to allow skipping label count validation on the HTTP write path based onX-Mimir-SkipLabelCountValidation
header beingtrue
or not. #9576 - [FEATURE] Ruler: Add experimental support for caching the contents of rule groups. This is disabled by default and can be enabled by setting
-ruler-storage.cache.rule-group-enabled
. #9595 #10024 - [FEATURE] PromQL: Add experimental
info
function. Experimental functions are disabled by default, but can be enabled setting-querier.promql-experimental-functions-enabled=true
in the query-frontend and querier. #9879 - [FEATURE] Distributor: Support promotion of OTel resource attributes to labels. #8271
- [FEATURE] Querier: Add experimental
double_exponential_smoothing
PromQL function. Experimental functions are disabled by default, but can be enabled by setting-querier.promql-experimental-functions-enabled=true
in the query-frontend and querier. #9844 - [ENHANCEMENT] Query Frontend: Return server-side
bytes_processed
statistics following Server-Timing format. #9645 #9985 - [ENHANCEMENT] mimirtool: Adds bearer token support for mimirtool's analyze ruler/prometheus commands. #9587
- [ENHANCEMENT] Ruler: Support
exclude_alerts
parameter in<prometheus-http-prefix>/api/v1/rules
endpoint. #9300 - [ENHANCEMENT] Distributor: add a metric to track tenants who are sending newlines in their label values called
cortex_distributor_label_values_with_newlines_total
. #9400 - [ENHANCEMENT] Ingester: improve performance of reading the WAL. #9508
- [ENHANCEMENT] Query-scheduler: improve the errors and traces emitted by query-schedulers when communicating with queriers. #9519
- [ENHANCEMENT] Compactor: uploaded blocks cannot be bigger than max configured compactor time range, and cannot cross the boundary for given time range. #9524
- [ENHANCEMENT] The distributor now validates that received label values only contain allowed characters. #9185
- [ENHANCEMENT] Add SASL plain authentication support to Kafka client used by the experimental ingest storage. Configure SASL credentials via the following settings: #9584
-ingest-storage.kafka.sasl-password
-ingest-storage.kafka.sasl-username
- [ENHANCEMENT] memberlist: TCP transport write path is now non-blocking, and is configurable by new flags: #9594
-memberlist.max-concurrent-writes
-memberlist.acquire-writer-timeout
- [ENHANCEMENT] memberlist: Notifications can now be processed once per interval specified by
-memberlist.notify-interval
to reduce notify storm CPU activity in large clusters. #9594 - [ENHANCEMENT] Query-scheduler: Remove the experimental
query-scheduler.prioritize-query-components
flag. Request queues always prioritize query component dequeuing above tenant fairness. #9703 - [ENHANCEMENT] Ingester: Emit traces for block syncing, to join up block-upload traces. #9656
- [ENHANCEMENT] Querier: Enable the optional querying of additional storage queryables. #9712
- [ENHANCEMENT] Ingester: Disable the push circuit breaker when ingester is in read-only mode. #9760
- [ENHANCEMENT] Ingester: Reduced lock contention in the
PostingsForMatchers
cache. #9773 - [ENHANCEMENT] Storage: Allow HTTP client settings to be tuned for GCS and Azure backends via an
http
block or corresponding CLI flags. This was already supported by the S3 backend. #9778 - [ENHANCEMENT] Ruler: Support
group_limit
andgroup_next_token
parameters in the<prometheus-http-prefix>/api/v1/rules
endpoint. #9563 - [ENHANCEMENT] Ingester: improved lock contention affecting read and write latencies during TSDB head compaction. #9822
- [ENHANCEMENT] Distributor: when a label value fails validation due to invalid UTF-8 characters, don't include the invalid characters in the returned error. #9828
- [ENHANCEMENT] Ingester: when experimental ingest storage is enabled, do not buffer records in the Kafka client when fetch concurrency is in use. #9838 #9850
- [ENHANCEMENT] Compactor: refresh deletion marks when updating the bucket index concurrently. This speeds up updating the bucket index by up to 16 times when there is a lot of blocks churn (thousands of blocks churning every cleanup cycle). #9881
- [ENHANCEMENT] PromQL: make
sort_by_label
stable. #9879 - [ENHANCEMENT] Distributor: Initialize ha_tracker cache before ha_tracker and distributor reach running state and begin serving writes. #9826 #9976
- [ENHANCEMENT] Ingester:
-ingest-storage.kafka.max-buffered-bytes
to limit the memory for buffered records when using concurrent fetching. #9892 - [ENHANCEMENT] Querier: improve performance and memory consumption of queries that select many series. #9914
- [ENHANCEMENT] Ruler: Support OAuth2 and proxies in Alertmanager client #9945 #10030
- [ENHANCEMENT] Ingester: Add
-blocks-storage.tsdb.bigger-out-of-order-blocks-for-old-samples
to build 24h blocks for out-of-order data belonging to the previous days instead of building smaller 2h blocks. This reduces pressure on compactors and ingesters when the out-of-order samples span multiple days in the past. #9844 #10033 #10035 - [ENHANCEMENT] Distributor: allow a different limit for info series (series ending in
_info
) label count, via-validation.max-label-names-per-info-series
. #10028 - [ENHANCEMENT] Ingester: do not reuse labels, samples and histograms slices in the write request if there are more entries than 10x the pre-allocated size. This should help to reduce the in-use memory in case of few requests with a very large number of labels, samples or histograms. #10040
- [ENHANCEMENT] Query-Frontend: prune
<subquery> and on() (vector(x)==y)
style queries and stop pruning<subquery> < -Inf
. Triggered by prometheus/prometheus#15245. #10026 - [ENHANCEMENT] Query-Frontend: perform request format validation before processing the request. #10093
- [BUGFIX] Fix issue where functions such as
rate()
over native histograms could return incorrect values if a float stale marker was present in the selected range. #9508 - [BUGFIX] Fix issue where negation of native histograms (eg.
-some_native_histogram_series
) did nothing. #9508 - [BUGFIX] Fix issue where
metric might not be a counter, name does not end in _total/_sum/_count/_bucket
annotation would be emitted even ifrate
orincrease
did not have enough samples to compute a result. #9508 - [BUGFIX] Fix issue where sharded queries could return annotations with incorrect or confusing position information. #9536
- [BUGFIX] Fix issue where downstream consumers may not generate correct cache keys for experimental error caching. #9644
- [BUGFIX] Fix issue where active series requests error when encountering a stale posting. #9580
- [BUGFIX] Fix pooling buffer reuse logic when
-distributor.max-request-pool-buffer-size
is set. #9666 - [BUGFIX] Fix issue when using the experimental
-ruler.max-independent-rule-evaluation-concurrency
feature, where the ruler could panic as it updates a running ruleset or shutdowns. #9726 - [BUGFIX] Always return unknown hint for first sample in non-gauge native histograms chunk to avoid incorrect counter reset hints when merging chunks from different sources. #10033
- [BUGFIX] Ensure native histograms counter reset hints are corrected when merging results from different sources. #9909
- [BUGFIX] Ingester: Fix race condition in per-tenant TSDB creation. #9708
- [BUGFIX] Ingester: Fix race condition in exemplar adding. #9765
- [BUGFIX] Ingester: Fix race condition in native histogram appending. #9765
- [BUGFIX] Ingester: Fix bug in concurrent fetching where a failure to list topics on startup would cause to use an invalid topic ID (0x00000000000000000000000000000000). #9883
- [BUGFIX] Ingester: Fix data loss bug in the experimental ingest storage when a Kafka Fetch is split into multiple requests and some of them return an error. #9963 #9964
- [BUGFIX] PromQL:
round
now removes the metric name again. #9879 - [BUGFIX] Query-Frontend: fix
QueryFrontendCodec
module initialization to set lookback delta from-querier.lookback-delta
. #9984 - [BUGFIX] OTLP: Support integer exemplar value type. #9844
- [BUGFIX] Querier: Correct the behaviour of binary operators between native histograms and floats. #9844
- [BUGFIX] Querier: Fix stddev+stdvar aggregations to always ignore native histograms. #9844
- [BUGFIX] Querier: Fix stddev+stdvar aggregations to treat Infinity consistently. #9844
- [BUGFIX] Ingester: Chunks could have one unnecessary zero byte at the end. #9844
- [BUGFIX] OTLP receiver: Preserve colons and combine multiple consecutive underscores into one when generating metric names in suffix adding mode (
-distributor.otel-metric-suffixes-enabled
). #10075
Mixin
- [CHANGE] Remove backwards compatibility for
thanos_memcached_
prefixed metrics in dashboards and alerts removed in 2.12. #9674 #9758 - [CHANGE] Reworked the alert
MimirIngesterStuckProcessingRecordsFromKafka
to also work when concurrent fetching is enabled. #9855 - [ENHANCEMENT] Unify ingester autoscaling panels on 'Mimir / Writes' dashboard to work for both ingest-storage and non-ingest-storage autoscaling. #9617
- [ENHANCEMENT] Alerts: Enable configuring job prefix for alerts to prevent clashes with metrics from Loki/Tempo. #9659
- [ENHANCEMENT] Dashboards: visualize the age of source blocks in the "Mimir / Compactor" dashboard. #9697
- [ENHANCEMENT] Dashboards: Include block compaction level on queried blocks in 'Mimir / Queries' dashboard. #9706
- [ENHANCEMENT] Alerts: add
MimirIngesterMissedRecordsFromKafka
to detect gaps in consumed records in the ingester when using the experimental Kafka-based storage. #9921 #9972 - [ENHANCEMENT] Dashboards: Add more panels to 'Mimir / Writes' for concurrent ingestion and fetching when using ingest storage. #10021
- [BUGFIX] Dashboards: Fix autoscaling metrics joins when series churn. #9412 #9450 #9432
- [BUGFIX] Alerts: Fix autoscaling metrics joins in
MimirAutoscalerNotActive
when series churn. #9412 - [BUGFIX] Alerts: Exclude failed cache "add" operations from alerting since failures are expected in normal operation. #9658
- [BUGFIX] Alerts: Exclude read-only replicas from
IngesterInstanceHasNoTenants
alert. #9843 - [BUGFIX] Alerts: Use resident set memory for the
EtcdAllocatingTooMuchMemory
alert so that ephemeral file cache memory doesn't cause the alert to misfire. #9997 - [BUGFIX] Query-frontend: support
X-Read-Consistency-Offsets
on labels queries too.
Jsonnet
- [CHANGE] Remove support to set Redis as a cache backend from jsonnet. #9677
- [CHANGE] Rollout-operator now defaults to storing scaling operation metadata in a Kubernetes ConfigMap. This avoids recursively invoking the admission webhook in some Kubernetes environments. #9699
- [CHANGE] Update rollout-operator version to 0.20.0. #9995
- [CHANGE] Remove the
track_sizes
feature for Memcached pods since it is unused. #10032 - [CHANGE] The configuration options
autoscaling_distributor_min_replicas
andautoscaling_distributor_max_replicas
has been renamed toautoscaling_distributor_min_replicas_per_zone
andautoscaling_distributor_max_replicas_per_zone
respectively. #10019 - [FEATURE] Add support to deploy distributors in multi availability zones. #9548
- [FEATURE] Add configuration settings to set the number of Memcached replicas for each type of cache (
memcached_frontend_replicas
,memcached_index_queries_replicas
,memcached_chunks_replicas
,memcached_metadata_replicas
). #9679 - [ENHANCEMENT] Add
ingest_storage_ingester_autoscaling_triggers
option to specify multiple triggers in ScaledObject created for ingest-store ingester autoscaling. #9422 - [ENHANCEMENT] Add
ingest_storage_ingester_autoscaling_scale_up_stabilization_window_seconds
andingest_storage_ingester_autoscaling_scale_down_stabilization_window_seconds
config options to make stabilization window for ingester autoscaling when using ingest-storage configurable. #9445 - [ENHANCEMENT] Make label-selector in ReplicaTemplate/ingester-zone-a object configurable when using ingest-storage. #9480
- [ENHANCEMENT] Add
querier_only_args
option to specify CLI flags that apply only to queriers but not ruler-queriers. #9503 - [ENHANCEMENT] Validate the Kafka client ID configured when ingest storage is enabled. #9573
- [ENHANCEMENT] Configure pod anti-affinity and tolerations to run etcd pods multi-AZ when
_config.multi_zone_etcd_enabled
is set totrue
. #9725
Mimirtool
Mimir Continuous Test
Query-tee
- [FEATURE] Added
-proxy.compare-skip-samples-before
to skip samples before the given time when comparing responses. The time can be in RFC3339 format (or) RFC3339 without the timezone and seconds (or) date only. #9515 - [FEATURE] Add
-backend.config-file
for a YAML configuration file for per-backend options. Currently, it only supports additional HTTP request headers. #10081 - [ENHANCEMENT] Added human-readable timestamps to comparison failure messages. #9665
Documentation
- [BUGFIX] Send native histograms: update the migration guide with the corrected dashboard query for switching between classic and native histograms queries. #10052
Tools
- [FEATURE]
splitblocks
: add new tool to split blocks larger than a specified duration into multiple blocks. #9517, #9779 - [ENHANCEMENT]
copyblocks
: add--skip-no-compact-block-duration-check
, which defaults tofalse
, to simplify targeting blocks that are not awaiting compaction. #9439 - [ENHANCEMENT]
copyblocks
: add--user-mapping
to support copying blocks between users. #10110 - [ENHANCEMENT]
kafkatool
: add SASL plain authentication support. The following new CLI flags have been added: #9584--kafka-sasl-username
--kafka-sasl-password
- [ENHANCEMENT]
kafkatool
: adddump print
command to print the content of write requests from a dump. #9942 - [ENHANCEMENT] Updated
KubePersistentVolumeFillingUp
runbook, including a sample command to debug the distroless image. #9802
All changes in this release: mimir-2.14.0-rc.0...mimir-2.15.0-rc.0