Skip to content

Conversation

@l0rinc
Copy link
Collaborator

@l0rinc l0rinc commented Jan 31, 2025

Just saving it here as an experiment that I'll continue later

2025-01-31T21:46:56Z BenchmarkLoadAllUTXOs: Loading all UTXOs from disk started
2025-01-31T21:47:46Z BenchmarkLoadAllUTXOs: Loading all UTXOs from disk completed (50.03s)
2025-01-31T21:47:46Z BenchmarkLoadAllUTXOs: Populating new cache with all UTXOs started
2025-01-31T21:52:57Z BenchmarkLoadAllUTXOs: Populating new cache with all UTXOs completed (310.82s)
2025-01-31T21:52:57Z BenchmarkLoadAllUTXOs: Loading UTXOs one-by-one started
2025-01-31T21:57:25Z BenchmarkLoadAllUTXOs: Loading UTXOs one-by-one completed (267.58s)
2025-01-31T21:57:25Z BenchmarkLoadAllUTXOs: Loading UTXOs using cursor started
2025-01-31T22:03:19Z BenchmarkLoadAllUTXOs: Loading UTXOs using cursor completed (354.26s)
2025-01-31T22:03:26Z BenchmarkLoadAllUTXOs: Loading shuffled UTXOs one-by-one started
2025-01-31T22:10:32Z BenchmarkLoadAllUTXOs: Loading shuffled UTXOs one-by-one completed (426.43s)
2025-01-31T22:10:32Z BenchmarkLoadAllUTXOs: Loading shuffled UTXOs using cursor started
2025-01-31T22:19:18Z BenchmarkLoadAllUTXOs: Loading shuffled UTXOs using cursor completed (525.78s)
2025-01-31T22:19:30Z BenchmarkLoadAllUTXOs: Loading ascending UTXOs one-by-one started
2025-01-31T22:23:35Z BenchmarkLoadAllUTXOs: Loading ascending UTXOs one-by-one completed (244.93s)
2025-01-31T22:23:35Z BenchmarkLoadAllUTXOs: Loading ascending UTXOs using cursor started
2025-01-31T22:29:24Z BenchmarkLoadAllUTXOs: Loading ascending UTXOs using cursor completed (349.90s)

After loading the UTXO set via assumeUTXO this can help debugging how to create a multiget

josibake and others added 16 commits January 24, 2025 08:59
introduce two commandline options for assumeutxo, specifically for
benchmarking. these commands are:

- pausebackgroundsync - an option lets the user pause the verification
                        of historical blocks in the background
-loadutxosnapshot=<path> - load an assumeutxo snapshot on startup,
                           instead of needing to go through the rpc
                           command. the node will shutdown immediately
                           after the snapshot has been loaded

this path is not meant for general use and is instead just for making it
more ergonomic to use assumeutxo for benchmarking IBD changes. the
benefits of using assumeutxo here are we can start from an arbitrary
height and sync to chaintip to collect relevant data quickly. using
assumeutxo means we can make whatever changes we need to the
chainstatedb, since it will be created fresh from the snapshot.

note, to use the loadutxosnapshot option, you must first run:

    ./build/src/bitcoind -stopatheight=1

this makes the node do a header sync and then shut down. this is because
assumeutxo will not load a snapshot unless the base block is in the
header chain. we could remove this requirement, but this patch is meant
to be as minimal as possible, and this also allows us to perform heaeder
sync as a preparation commit for a benchmark, which helps keep IBD
benchmarks more focused on strictly measuring IBD.

next, run:

    ./build/src/bitcoind -loadutxosnapshot=<path>

the node will shutdown after the snapshot is loaded. finally, run:

    ./build/src/bitcoind -pausebackgroundsync=1

for the actual benchmarking step. this ensures only the sync to chaintip
is benchmarked and the load snapshot step is not included in the
measurement.

Co-authored-by: Sjors Provoost <[email protected]>
Co-authored-by: David Gumberg <[email protected]>
Co-authored-by: Lőrinc <[email protected]>
After loading the UTXO set via assumeUTXO this can help debugging how to create a multiget
@willcl-ark willcl-ark force-pushed the master branch 4 times, most recently from 91555c9 to d07e4c3 Compare February 4, 2025 15:13
@willcl-ark willcl-ark force-pushed the master branch 5 times, most recently from d15ce87 to 4203957 Compare February 12, 2025 21:23
@willcl-ark willcl-ark force-pushed the master branch 4 times, most recently from 54b2281 to e85040f Compare February 13, 2025 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants