Description of Changes

This patch integrates the new query engine into the query code path. It does not integrate the new engine into incremental evaluation path.

What is implemented?

1. Instantiating tuple-at-a-time iterators from a physical plan
2. The addition of a new query crate which is the top level crate for invoking the query engine.
3. Invoking the new engine for initial subscriptions and one off queries
4. Benchmarks

What isn't implemented?

1. Incremental evaluation

The old engine will be removed once incremental evaluation has been updated to use the new engine.

API and ABI breaking changes

n/a

Expected complexity level and risk

4

The query engine is responsible for implementing the subscription logic, which is the only way to read data out of SpacetimeDB. As such, its correctness and performance is paramount.

Testing

Does the new engine preserve correctness? Does it also preserve performance?

We have already added tests to ensure we preserve semantics, with more being worked on currently. This patch updates the applicable benchmarks to ensure we maintain performance characteristics.

full-scan               time:   [10.988 ms 11.039 ms 11.098 ms]
                        change: [-24.628% -24.131% -23.622%] (p = 0.00 < 0.05)
                        Performance has improved.

full-join               time:   [92.660 µs 92.817 µs 92.976 µs]
                        change: [-34.859% -34.730% -34.595%] (p = 0.00 < 0.05)
                        Performance has improved.

query-indexes-multi     time:   [361.15 ns 361.59 ns 362.10 ns]
                        change: [-30.401% -30.266% -30.125%] (p = 0.00 < 0.05)
                        Performance has improved.