You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The BufferFilter and downstream code that uses it when indexing the buffer could be simplified by removing the handling of NotIn guarantees.
They provide little value, expecially if they just end up selecting most of what is in the buffer - at that point, we might as well just let DataFusion do the filtering.
Proposed solution
Skip NotIn guarantees when processing the results of LiteralGuarantee::analyze in the BufferFilter::new method:
Problem statement
The
BufferFilter
and downstream code that uses it when indexing the buffer could be simplified by removing the handling ofNotIn
guarantees.They provide little value, expecially if they just end up selecting most of what is in the buffer - at that point, we might as well just let DataFusion do the filtering.
Proposed solution
Skip
NotIn
guarantees when processing the results ofLiteralGuarantee::analyze
in theBufferFilter::new
method:influxdb/influxdb3_write/src/lib.rs
Lines 583 to 589 in b9a7927
This will break some tests, so the tests in
table_buffer.rs
will need to be cleaned up, as sell as the logic that is handling theNotIn
guarantees.Additional context
See #25866 (comment)
The text was updated successfully, but these errors were encountered: