Skip to content

Conversation

@zhuqi-lucas
Copy link
Collaborator

@zhuqi-lucas zhuqi-lucas commented Oct 28, 2025

Before we optimize the many columns cases, we adding the heavy operation to benchmark.

cargo bench --bench materialized_views_benchmark
   Compiling datafusion-materialized-views v0.1.1 (/Users/zhuqi/polygon/datafusion-materialized-views)
    Finished `bench` profile [optimized] target(s) in 13.95s
     Running benches/materialized_views_benchmark.rs (target/release/deps/materialized_views_benchmark-a0e1fcbb7898483b)
view_matcher_spj/spj_normal_form_new/cols=1
                        time:   [909.49 ns 912.06 ns 914.64 ns]
                        thrpt:  [1.0933 Melem/s 1.0964 Melem/s 1.0995 Melem/s]
                 change:
                        time:   [-0.7501% -0.1503% +0.3854%] (p = 0.62 > 0.05)
                        thrpt:  [-0.3839% +0.1505% +0.7558%]
                        No change in performance detected.
Found 1 outliers among 30 measurements (3.33%)
  1 (3.33%) low mild
view_matcher_spj/spj_normal_form_new_query/cols=1
                        time:   [906.39 ns 911.92 ns 918.96 ns]
                        thrpt:  [1.0882 Melem/s 1.0966 Melem/s 1.1033 Melem/s]
                 change:
                        time:   [+0.0673% +1.0194% +1.9644%] (p = 0.03 < 0.05)
                        thrpt:  [-1.9266% -1.0091% -0.0672%]
                        Change within noise threshold.
Found 4 outliers among 30 measurements (13.33%)
  1 (3.33%) low mild
  3 (10.00%) high mild
view_matcher_spj/rewrite_from/cols=1
                        time:   [2.3963 µs 2.4007 µs 2.4053 µs]
                        thrpt:  [415.76 Kelem/s 416.54 Kelem/s 417.31 Kelem/s]
                 change:
                        time:   [+1.2817% +1.8192% +2.4516%] (p = 0.00 < 0.05)
                        thrpt:  [-2.3929% -1.7867% -1.2654%]
                        Performance has regressed.
Found 5 outliers among 30 measurements (16.67%)
  1 (3.33%) low mild
  2 (6.67%) high mild
  2 (6.67%) high severe
view_matcher_spj/spj_normal_form_new/cols=10
                        time:   [4.4497 µs 4.4734 µs 4.4979 µs]
                        thrpt:  [222.33 Kelem/s 223.54 Kelem/s 224.73 Kelem/s]
                 change:
                        time:   [-0.8124% -0.4041% +0.0062%] (p = 0.06 > 0.05)
                        thrpt:  [-0.0062% +0.4057% +0.8191%]
                        No change in performance detected.
view_matcher_spj/spj_normal_form_new_query/cols=10
                        time:   [4.0019 µs 4.0126 µs 4.0237 µs]
                        thrpt:  [248.53 Kelem/s 249.21 Kelem/s 249.88 Kelem/s]
                 change:
                        time:   [-1.7201% -0.9923% -0.3384%] (p = 0.01 < 0.05)
                        thrpt:  [+0.3396% +1.0022% +1.7502%]
                        Change within noise threshold.
Found 2 outliers among 30 measurements (6.67%)
  1 (3.33%) low mild
  1 (3.33%) high mild
view_matcher_spj/rewrite_from/cols=10
                        time:   [4.7247 µs 4.7322 µs 4.7408 µs]
                        thrpt:  [210.93 Kelem/s 211.32 Kelem/s 211.65 Kelem/s]
                 change:
                        time:   [-0.9537% -0.4327% +0.0066%] (p = 0.08 > 0.05)
                        thrpt:  [-0.0066% +0.4346% +0.9629%]
                        No change in performance detected.
Found 3 outliers among 30 measurements (10.00%)
  1 (3.33%) low mild
  2 (6.67%) high mild
view_matcher_spj/spj_normal_form_new/cols=20
                        time:   [8.6014 µs 8.6198 µs 8.6370 µs]
                        thrpt:  [115.78 Kelem/s 116.01 Kelem/s 116.26 Kelem/s]
                 change:
                        time:   [-0.0986% +0.5545% +1.1445%] (p = 0.08 > 0.05)
                        thrpt:  [-1.1315% -0.5514% +0.0987%]
                        No change in performance detected.
Found 2 outliers among 30 measurements (6.67%)
  2 (6.67%) high mild
view_matcher_spj/spj_normal_form_new_query/cols=20
                        time:   [9.4032 µs 9.4153 µs 9.4270 µs]
                        thrpt:  [106.08 Kelem/s 106.21 Kelem/s 106.35 Kelem/s]
                 change:
                        time:   [-2.2903% -0.3974% +0.7541%] (p = 0.77 > 0.05)
                        thrpt:  [-0.7485% +0.3989% +2.3440%]
                        No change in performance detected.
Found 1 outliers among 30 measurements (3.33%)
  1 (3.33%) high mild
view_matcher_spj/rewrite_from/cols=20
                        time:   [8.9988 µs 9.0244 µs 9.0496 µs]
                        thrpt:  [110.50 Kelem/s 110.81 Kelem/s 111.13 Kelem/s]
                 change:
                        time:   [-1.1601% -0.7346% -0.3378%] (p = 0.00 < 0.05)
                        thrpt:  [+0.3390% +0.7400% +1.1737%]
                        Change within noise threshold.
view_matcher_spj/spj_normal_form_new/cols=40
                        time:   [16.671 µs 16.742 µs 16.808 µs]
                        thrpt:  [59.496 Kelem/s 59.730 Kelem/s 59.985 Kelem/s]
Found 6 outliers among 30 measurements (20.00%)
  1 (3.33%) low severe
  1 (3.33%) low mild
  4 (13.33%) high mild
view_matcher_spj/spj_normal_form_new_query/cols=40
                        time:   [18.334 µs 18.398 µs 18.456 µs]
                        thrpt:  [54.184 Kelem/s 54.355 Kelem/s 54.543 Kelem/s]
Found 2 outliers among 30 measurements (6.67%)
  2 (6.67%) high mild
view_matcher_spj/rewrite_from/cols=40
                        time:   [20.073 µs 20.135 µs 20.208 µs]
                        thrpt:  [49.486 Kelem/s 49.666 Kelem/s 49.819 Kelem/s]
Found 3 outliers among 30 measurements (10.00%)
  1 (3.33%) high mild
  2 (6.67%) high severe
view_matcher_spj/spj_normal_form_new/cols=80
                        time:   [35.393 µs 35.559 µs 35.679 µs]
                        thrpt:  [28.028 Kelem/s 28.122 Kelem/s 28.254 Kelem/s]
Found 1 outliers among 30 measurements (3.33%)
  1 (3.33%) low mild
view_matcher_spj/spj_normal_form_new_query/cols=80
                        time:   [40.264 µs 40.388 µs 40.538 µs]
                        thrpt:  [24.668 Kelem/s 24.760 Kelem/s 24.836 Kelem/s]
view_matcher_spj/rewrite_from/cols=80
                        time:   [54.357 µs 54.481 µs 54.624 µs]
                        thrpt:  [18.307 Kelem/s 18.355 Kelem/s 18.397 Kelem/s]
Found 1 outliers among 30 measurements (3.33%)
  1 (3.33%) low mild
view_matcher_spj/spj_normal_form_new/cols=160
diff --git a/Cargo.toml b/Cargo.toml
index 6dea53b..431ea31 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -53,3 +53,9 @@ env_logger = "0.11.6"
 tempfile = "3.14.0"
 tokio = "1.42.0"
 url = "2.5.4"
+criterion = "0.4"
+
+[[bench]]
+name = "materialized_views_benchmark"
+harness = false
+path = "benches/materialized_views_benchmark.rs"
\ No newline at end of file
diff --git a/benches/materialized_views_benchmark.rs b/benches/materialized_views_benchmark.rs
index 4765736..1645b2b 100644
--- a/benches/materialized_views_benchmark.rs
+++ b/benches/materialized_views_benchmark.rs
@@ -1,20 +1,16 @@
 // benches/materialized_views_benchmark
                        time:   [73.943 µs 74.593 µs 75.244 µs]
                        thrpt:  [13.290 Kelem/s 13.406 Kelem/s 13.524 Kelem/s]
Found 3 outliers among 30 measurements (10.00%)
  3 (10.00%) high mild
view_matcher_spj/spj_normal_form_new_query/cols=160
                        time:   [91.300 µs 91.551 µs 91.805 µs]
                        thrpt:  [10.893 Kelem/s 10.923 Kelem/s 10.953 Kelem/s]
Found 5 outliers among 30 measurements (16.67%)
  1 (3.33%) low mild
  2 (6.67%) high mild
  2 (6.67%) high severe
view_matcher_spj/rewrite_from/cols=160
                        time:   [170.10 µs 170.47 µs 170.89 µs]
                        thrpt:  [5.8517 Kelem/s 5.8663 Kelem/s 5.8789 Kelem/s]
view_matcher_spj/spj_normal_form_new/cols=320
                        time:   [165.73 µs 166.47 µs 167.32 µs]
                        thrpt:  [5.9765 Kelem/s 6.0072 Kelem/s 6.0340 Kelem/s]
view_matcher_spj/spj_normal_form_new_query/cols=320
                        time:   [216.99 µs 218.08 µs 219.16 µs]
                        thrpt:  [4.5629 Kelem/s 4.5855 Kelem/s 4.6084 Kelem/s]
view_matcher_spj/rewrite_from/cols=320
                        time:   [584.00 µs 589.78 µs 597.42 µs]
                        thrpt:  [1.6739 Kelem/s 1.6955 Kelem/s 1.7123 Kelem/s]
Found 3 outliers among 30 measurements (10.00%)
  3 (10.00%) high severe

@zhuqi-lucas zhuqi-lucas merged commit 330f367 into branch-50 Oct 28, 2025
8 checks passed
@github-actions github-actions bot mentioned this pull request Jun 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants