Skip to content

[WIP] feat(flink): Support bootstrap from RLI to local RocksDB for flink bu…#18254

Open
cshuo wants to merge 3 commits intoapache:masterfrom
cshuo:blocking_rli_bootstrap
Open

[WIP] feat(flink): Support bootstrap from RLI to local RocksDB for flink bu…#18254
cshuo wants to merge 3 commits intoapache:masterfrom
cshuo:blocking_rli_bootstrap

Conversation

@cshuo
Copy link
Collaborator

@cshuo cshuo commented Feb 26, 2026

…cket assigner

  • Support scan all record index locations
  • Support RocksDb index backend.
  • Add a bootstrap operator for bootstraping RLI into RocksDB index backend.
  • Blocking rli bootstrap on job restart/failover until all pending instants being committed successfully.

Describe the issue this Pull Request addresses

Summary and Changelog

Impact

Risk Level

Documentation Update

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

…cket assigner

* Support scan all record index locations
* Support RocksDb index backend.
* Add a bootstrap operator for bootstraping RLI into RocksDB index backend.
* Blocking rli bootstrap on job restart/failover until all pending instants being committed successfully.
@github-actions github-actions bot added the size:L PR with lines of changes in (300, 1000] label Feb 26, 2026
update default value for fg number of RLI partition
@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 0% with 50 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.28%. Comparing base (fb7b1a5) to head (e8c53f9).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
...pache/hudi/metadata/HoodieBackedTableMetadata.java 0.00% 49 Missing ⚠️
.../org/apache/hudi/metadata/HoodieTableMetadata.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18254      +/-   ##
============================================
- Coverage     57.30%   57.28%   -0.02%     
- Complexity    18561    18574      +13     
============================================
  Files          1945     1947       +2     
  Lines        106256   106446     +190     
  Branches      13131    13157      +26     
============================================
+ Hits          60885    60973      +88     
- Misses        39648    39727      +79     
- Partials       5723     5746      +23     
Flag Coverage Δ
hadoop-mr-java-client 45.31% <0.00%> (-0.09%) ⬇️
spark-java-tests 47.44% <0.00%> (+0.01%) ⬆️
spark-scala-tests 45.51% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
.../org/apache/hudi/metadata/HoodieTableMetadata.java 65.00% <0.00%> (-1.67%) ⬇️
...pache/hudi/metadata/HoodieBackedTableMetadata.java 70.20% <0.00%> (-10.21%) ⬇️

... and 20 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L PR with lines of changes in (300, 1000]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants