feat(flashblocks): add state trie cache warming for witness generation

[niran]

Overview

Add background state trie cache warming feature that calculates state roots to pre-warm state trie caches before witness generation. This addresses performance issues when state root calculation is disabled.

⚠️ DRAFT PR - This is an untested proof-of-concept to illustrate potential performance improvements that might help Base avoid missed blocks.

Problem Statement

When the builder runs with disable_state_root: true, witness generation for the state trie uses cold caches, causing slow performance (~500-2000ms). This can contribute to missed blocks during high-load periods.

Solution

Proactively calculate state roots in the background after each flashblock (even though the result isn't used) to warm state trie caches. The warming process runs asynchronously and the warm caches remain available for fast witness generation.

Key Features

• Background warming: State root calculation runs in background threads after each flashblock publishes, never blocking the main build pipeline
• Smart throttling: Only one warming task runs at a time using atomic flags; additional attempts are skipped and tracked in metrics
• Non-interruptible: State root calculation runs to completion once started, but doesn't block new FCU arrivals
• Comprehensive metrics: Full observability with start/complete/skip/error counters and duration histograms

Configuration

New CLI flag and environment variable:

--flashblocks.enable-state-trie-warming  # (default: false)
FLASHBLOCKS_ENABLE_STATE_TRIE_WARMING=true

Expected Performance Impact

• Cold cache state root: ~500-2000ms (disk I/O intensive)
• Warm cache state root: ~50-200ms (CPU-bound)
• Net improvement: 5-10x faster witness generation
• Use case: Helps avoid missed blocks during high-load periods

Implementation Details

1. Added CLI flag to FlashblocksArgs (crates/builder/base-builder-cli/src/flashblocks.rs:45-52)
2. Added config field to FlashblocksConfig (crates/builder/op-rbuilder/src/flashblocks/config.rs:40)
3. Created StateTrieWarmer module with background task spawning (crates/builder/op-rbuilder/src/flashblocks/state_trie_warmer.rs)
4. Added 5 new metrics for observability (crates/builder/op-rbuilder/src/metrics.rs:187-196)
5. Integrated warming after each flashblock publishes (crates/builder/op-rbuilder/src/flashblocks/payload.rs:576-579)
6. Warming uses spawn_blocking for CPU-intensive work

Metrics

Monitor the feature with:

• op_rbuilder_state_trie_warming_started_count - Tasks started
• op_rbuilder_state_trie_warming_completed_count - Tasks completed successfully
• op_rbuilder_state_trie_warming_skipped_count - Tasks skipped (already warming)
• op_rbuilder_state_trie_warming_duration - Duration histogram
• op_rbuilder_state_trie_warming_error_count - Error count

Testing Status

❌ NOT TESTED - This implementation has not been tested in any environment (dev, staging, or production).

Next Steps

Before considering for production:

1. Test in development environment
2. Validate metrics are being recorded correctly
3. Measure actual performance impact on witness generation
4. Verify no negative impact on flashblock building
5. Test under high load conditions
6. Verify behavior with rapid FCU arrivals
7. Consider CPU usage impact of background warming

Files Changed

• crates/builder/base-builder-cli/src/flashblocks.rs - CLI flag
• crates/builder/op-rbuilder/src/flashblocks/config.rs - Config field
• crates/builder/op-rbuilder/src/flashblocks/state_trie_warmer.rs - Core warming logic (new file)
• crates/builder/op-rbuilder/src/flashblocks/payload.rs - Integration point
• crates/builder/op-rbuilder/src/flashblocks/mod.rs - Module exports
• crates/builder/op-rbuilder/src/metrics.rs - New metrics

[cb-heimdall]

🟡 Heimdall Review Status

Requirement	Status	More Info
Reviews	🟡 0/1	Denominator calculation Show calculation 1 if user is bot 0 1 if user is external 0 2 if repo is sensitive 0 From .codeflow.yml 1 Additional review requirements Show calculation Max 0 0 From CODEOWNERS 0 Global minimum 0 Max 1 1 1 if commit is unverified 0 Sum 1