Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

release-23.2: bump Pebble and split snapshot sstables #134526

Merged
merged 3 commits into from
Nov 7, 2024
Merged

release-23.2: bump Pebble and split snapshot sstables #134526

merged 3 commits into from
Nov 7, 2024

Commits on Nov 7, 2024

  1. go.mod: bump Pebble to 19b47dc92d30 

    cockroachdb/pebble@19b47dc9 db: avoid loading later files for a level in SeekPrefixGE
cockroachdb/pebble@74498653 db: export MakeTrailer
cockroachdb/pebble@96bee8ff internal/keyspan: export Truncate in keyspan.Fragmenter.

Release note (performance improvement): Avoid loading unnecessary file blocks shortly
after a rebalance in a rare case.

Epic: none
    @itsbilal
    itsbilal committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    253d94e View commit details
    Browse the repository at this point in the history

  2. kvserver: split snapshot SSTables for mvcc keys into multiple SSTs 

    Previously, we'd only create one sstable for all mvcc keys
in a range when ingesting a rebalance/recovery snapshot into
Pebble. This increased write-amp in Pebble as more sstables
would have to be compacted into it (or the sstable then split
into smaller ones in Pebble), and had other consequences
such as massive filter blocks in the large singular sstable.

This change adds a new cluster setting,
kv.snapshot_rebalance.max_sst_size, that sets the max size of the
sstables containing user/mvcc keys in a range. If an sstable exceeds
this size in multiSSTWriter, we roll over that sstable and create a
new one.

Epic: CRDB-8471
Fixes: cockroachdb#67284

Release note (performance improvement): Reduce the write-amplification
impact of rebalances by splitting snapshot sstable files into smaller ones
before ingesting them into Pebble.
    @itsbilal
    itsbilal committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    b9362d0 View commit details
    Browse the repository at this point in the history

  3. kvserver: reenable splitting of snapshot sstables 

    This change updates the snapshot strategy's sender side
to iterate over points and ranges together, instead of only
iterating on points first, then only ranges. This allows us to
more efficiently split snapshot sstables on the receiver side.
To avoid the need to add a version gate on the receiver side, we
propagate a bool, RangeKeysInOrder, to the receiver which is a signal
to it to enable sstable splits.

Fixes cockroachdb#129026.

Epic: none

Release note: None
    @itsbilal
    itsbilal committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    e17115b View commit details
    Browse the repository at this point in the history