Skip to content

Conversation

@DrakeLin
Copy link
Collaborator

@DrakeLin DrakeLin commented Jan 21, 2026

🥞 Stacked PR

Use this link to review incremental changes.


What changes are proposed in this pull request?

Adds a new stats_transform module that provides the core logic for populating stats and stats_parsed fields when writing checkpoints. This module builds transform expressions based on table configuration to ensure statistics are properly converted between JSON and struct formats.

writeStatsAsJson writeStatsAsStruct stats stats_parsed
true false COALESCE(stats, ToJson(stats_parsed)) drop
true true COALESCE(stats, ToJson(stats_parsed)) COALESCE(stats_parsed, ParseJson(stats))
false true drop COALESCE(stats_parsed, ParseJson(stats))
false false drop drop

How was this change tested?

Next Pr

@codecov
Copy link

codecov bot commented Jan 21, 2026

Codecov Report

❌ Patch coverage is 83.12883% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.11%. Comparing base (435b94e) to head (eab701e).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
kernel/src/checkpoint/stats_transform.rs 78.04% 54 Missing ⚠️
...src/engine/arrow_expression/evaluate_expression.rs 94.44% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1646      +/-   ##
==========================================
- Coverage   84.15%   84.11%   -0.04%     
==========================================
  Files         123      124       +1     
  Lines       34180    34498     +318     
  Branches    34180    34498     +318     
==========================================
+ Hits        28764    29019     +255     
- Misses       4021     4070      +49     
- Partials     1395     1409      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions github-actions bot added the breaking-change Change that require a major version bump label Jan 21, 2026
@DrakeLin DrakeLin force-pushed the stack/checkpoint-transforms branch from af4261b to 0aefe2a Compare January 21, 2026 00:21
@DrakeLin DrakeLin marked this pull request as ready for review January 21, 2026 00:25
@DrakeLin DrakeLin requested review from dengsh12 and nicklan January 21, 2026 00:25
@DrakeLin DrakeLin force-pushed the stack/checkpoint-transforms branch from 0aefe2a to 628c05b Compare January 21, 2026 00:37
@github-actions github-actions bot removed the breaking-change Change that require a major version bump label Jan 21, 2026
@DrakeLin DrakeLin force-pushed the stack/checkpoint-transforms branch from 628c05b to 66c1b43 Compare January 21, 2026 07:24
@DrakeLin DrakeLin force-pushed the stack/checkpoint-transforms branch from 66c1b43 to 55236f5 Compare January 21, 2026 07:51
@DrakeLin DrakeLin force-pushed the stack/checkpoint-transforms branch from 55236f5 to eab701e Compare January 21, 2026 23:38
@github-actions github-actions bot added the breaking-change Change that require a major version bump label Jan 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-change Change that require a major version bump

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant