Skip to content

leveldb max file size regression #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
davidgumberg wants to merge 12 commits into
base: master
Choose a base branch
from
Open

leveldb max file size regression #144

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
3782f3f
bench: add shell.nix
willcl-ark Feb 10, 2025
01d3035
bench: add uv + python deps
willcl-ark Feb 10, 2025
0d9071c
bench: add benchmark ci workflows
willcl-ark Feb 10, 2025
dfd2971
guix: build static
willcl-ark Feb 10, 2025
9cec269
doc: add benchcoin docs
willcl-ark Feb 10, 2025
7691ec0
remove legacy assumeutxo bench
willcl-ark Feb 13, 2025
1d161a8
use *instrumented for flame runs
willcl-ark Feb 13, 2025
6d960ca
add uninstrumented run
willcl-ark Feb 13, 2025
96e31f0
include instrumentation in name to avoid conflicts
willcl-ark Feb 14, 2025
10fe259
fixup! include instrumentation in name to avoid conflicts
willcl-ark Feb 14, 2025
6713bd1
allow failing source guix profile
willcl-ark Feb 14, 2025
19d948e
Performance regression: max file size back to 2 MiB default
davidgumberg Mar 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../doc/benchcoin.md
177 changes: 177 additions & 0 deletions .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
name: Benchmark
on:
pull_request:
branches:
- master
jobs:
build-binaries:
runs-on: [self-hosted, linux, x64]
env:
NIX_PATH: nixpkgs=channel:nixos-unstable
BASE_SHA: ${{ github.event.pull_request.base.sha }}
steps:
- name: Checkout repo
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Fetch base commit
run: |
echo "CHECKOUT_COMMIT=$(git rev-parse HEAD)" >> "$GITHUB_ENV"
git fetch --depth=1 origin ${{ github.event.pull_request.base.sha }}
- name: Setup ccache
run: |
mkdir -p /data/ccache
export CCACHE_DIR=/data/ccache
export CCACHE_MAXSIZE=50G
ccache -M 50G
ccache -s
- name: Build both binaries
env:
CCACHE_DIR: /data/ccache
run: |
mkdir -p ${{ runner.temp }}/binaries/base
mkdir -p ${{ runner.temp }}/binaries/head
nix-shell --command "just build-assumeutxo-binaries-guix $BASE_SHA $CHECKOUT_COMMIT"
cp binaries/base/bitcoind ${{ runner.temp }}/binaries/base/bitcoind
cp binaries/head/bitcoind ${{ runner.temp }}/binaries/head/bitcoind
- name: Upload binaries
uses: actions/upload-artifact@v4
with:
name: bitcoind-binaries
path: ${{ runner.temp }}/binaries/
uninstrumented:
needs: build-binaries
strategy:
matrix:
include:
- network: mainnet
name: mainnet-default-uninstrumented
timeout: 600
datadir_path: /data/pruned-840k
dbcache: 450
- network: mainnet
name: mainnet-large-uninstrumented
timeout: 600
datadir_path: /data/pruned-840k
dbcache: 32000
runs-on: [self-hosted, linux, x64]
timeout-minutes: ${{ matrix.timeout }}
env:
NIX_PATH: nixpkgs=channel:nixos-unstable
ORIGINAL_DATADIR: ${{ matrix.datadir_path }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
steps:
- name: Checkout repo
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Download binaries
uses: actions/download-artifact@v4
with:
name: bitcoind-binaries
path: ${{ runner.temp }}/binaries
- name: Set binary permissions
run: |
chmod +x ${{ runner.temp }}/binaries/base/bitcoind
chmod +x ${{ runner.temp }}/binaries/head/bitcoind
- name: Fetch base commit
run: |
echo "CHECKOUT_COMMIT=$(git rev-parse HEAD)" >> "$GITHUB_ENV"
git fetch --depth=1 origin ${{ github.event.pull_request.base.sha }}
- name: Run AssumeUTXO ${{ matrix.network }}
env:
TMP_DATADIR: "${{ runner.temp }}/base_datadir"
BINARIES_DIR: "${{ runner.temp }}/binaries"
run: |
env
mkdir -p "$TMP_DATADIR"
nix-shell --command "just run-${{ matrix.network }}-ci $BASE_SHA $CHECKOUT_COMMIT $TMP_DATADIR $ORIGINAL_DATADIR ${{ runner.temp }}/results.json ${{ matrix.dbcache }} $BINARIES_DIR"
- uses: actions/upload-artifact@v4
with:
name: result-${{ matrix.name }}
path: "${{ runner.temp }}/results.json"
- name: Write GitHub and runner context files
env:
GITHUB_CONTEXT: ${{ toJSON(github) }}
RUNNER_CONTEXT: ${{ toJSON(runner) }}
run: |
mkdir contexts
echo "$GITHUB_CONTEXT" | nix-shell -p jq --command "jq 'del(.token)' > contexts/github.json"
echo "$RUNNER_CONTEXT" > contexts/runner.json
- name: Upload context metadata as artifact
uses: actions/upload-artifact@v4
with:
name: run-metadata-${{ matrix.name }}
path: ./contexts/
instrumented:
needs: build-binaries
strategy:
matrix:
include:
- network: mainnet
name: mainnet-default-instrumented
timeout: 600
datadir_path: /data/pruned-840k
dbcache: 450
- network: mainnet
name: mainnet-large-instrumented
timeout: 600
datadir_path: /data/pruned-840k
dbcache: 32000
runs-on: [self-hosted, linux, x64]
timeout-minutes: ${{ matrix.timeout }}
env:
NIX_PATH: nixpkgs=channel:nixos-unstable
ORIGINAL_DATADIR: ${{ matrix.datadir_path }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
steps:
- name: Checkout repo
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Download binaries
uses: actions/download-artifact@v4
with:
name: bitcoind-binaries
path: ${{ runner.temp }}/binaries
- name: Set binary permissions
run: |
chmod +x ${{ runner.temp }}/binaries/base/bitcoind
chmod +x ${{ runner.temp }}/binaries/head/bitcoind
- name: Fetch base commit
run: |
echo "CHECKOUT_COMMIT=$(git rev-parse HEAD)" >> "$GITHUB_ENV"
git fetch --depth=1 origin ${{ github.event.pull_request.base.sha }}
- name: Run AssumeUTXO ${{ matrix.network }}
env:
TMP_DATADIR: "${{ runner.temp }}/base_datadir"
BINARIES_DIR: "${{ runner.temp }}/binaries"
run: |
env
mkdir -p "$TMP_DATADIR"
nix-shell --command "just run-${{ matrix.network }}-ci-instrumented $BASE_SHA $CHECKOUT_COMMIT $TMP_DATADIR $ORIGINAL_DATADIR ${{ runner.temp }}/results.json ${{ matrix.dbcache }} ${{ runner.temp }}/pngs $BINARIES_DIR"
- uses: actions/upload-artifact@v4
with:
name: result-${{ matrix.name }}
path: "${{ runner.temp }}/results.json"
- uses: actions/upload-artifact@v4
with:
name: pngs-${{ matrix.name }}
path: "${{ runner.temp }}/pngs/*.png"
- uses: actions/upload-artifact@v4
with:
name: flamegraph-${{ matrix.name }}
path: "**/*-flamegraph.svg"
- name: Write GitHub and runner context files
env:
GITHUB_CONTEXT: ${{ toJSON(github) }}
RUNNER_CONTEXT: ${{ toJSON(runner) }}
run: |
mkdir contexts
echo "$GITHUB_CONTEXT" | nix-shell -p jq --command "jq 'del(.token)' > contexts/github.json"
echo "$RUNNER_CONTEXT" > contexts/runner.json
- name: Upload context metadata as artifact
uses: actions/upload-artifact@v4
with:
name: run-metadata-${{ matrix.name }}
path: ./contexts/
Loading