MQE: track number of processed samples in each query #10232
Conversation
charleskorn
force-pushed
the
charleskorn/read-samples-tracking
branch
from
24164eb to
344bfc7
Compare
| @@ -119,6 +120,8 @@ func (v *InstantVectorSelector) NextSeries(ctx context.Context) (types.InstantVe | |||
| continue | |||
| } | |||
|
|
|||
| v.Stats.TotalSamples++ | |||
There was a problem hiding this comment.
Have we benchmarked this change?
One alternate would be to do v.Stats.TotalSamples += v.Selector.TimeRange.EndT / v.Selector.TimeRange.IntervalMilliseconds (probably with floor and an offset for the start or similar)
Or is this done here to catch stats for errored queries? I'm not sure if there would be value in that.
If not, alternatively we could look at the len of the returned data.
| // QueryStats tracks statistics about the execution of a single query. | ||
| // | ||
| // It is not safe to use this type from multiple goroutines simultaneously. | ||
| type QueryStats struct { |
There was a problem hiding this comment.
Why create our own struct instead of using stats.QuerySamples and passing that to the appropriate operators?
It would also make it easier to implement TotalSamplesPerStep if we want to support that too (which I think we do)
| dense_series 0 1 2 3 4 5 6 7 8 9 10 | ||
| start_series 0 1 _ _ _ _ _ _ _ _ _ | ||
| end_series _ _ _ _ _ 5 6 7 8 9 10 | ||
| sparse_series 0 _ _ _ _ _ _ 7 _ _ _ |
There was a problem hiding this comment.
should have some NH's to test too
Also some stale and NaN's etc.
There was a problem hiding this comment.
While addressing this I found out NH's were not being counted to samples as PromQL does. I have adjusted the calculation.
| require.Equal(t, testCase.expectedTotalSamples, prometheusCount, "invalid test case: expected samples does not match value from Prometheus' engine") | ||
|
|
||
| mimirCount := runQueryAndGetTotalSamples(t, mimirEngine, testCase.expr, testCase.isInstantQuery) | ||
| require.Equal(t, testCase.expectedTotalSamples, mimirCount) |
There was a problem hiding this comment.
We can also compare the samples loaded as part of our test gauntlet if we expect it to be the same in all cases
tinitiuset
force-pushed
the
charleskorn/read-samples-tracking
branch
from
56ffb11 to
5dea7f9
Compare
What this PR does
This PR adds support for tracking the number of samples processed in a query evaluated by MQE.
Which issue(s) this PR fixes or relates to
Resolves #10138
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX].about-versioning.mdupdated with experimental features.