Integrate fork choice compliance test suite #8419
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Description
Adds support of the fork choice compliance test suite, the source of which can be found in the following spec PR:
Running tests
To run tests one need to download one of the test corpus (differentiated by a number of tests):
Extract with command:
And use
ManualReferenceTestRunner
with-Dteku.ref-test-module.override-root=~/Downloads/tst_standard/
to specify the directory with the test vectors.It is required to expand the
consensus-spec-tests
suites to build the module whereManualReferenceTestRunner
is located so running the new tests this way looks more like a temporary solution and potentially a separate test runner is required.Implementation detail
The new test suite is based on the fork choice test suite and extends the fork choice test format with only one check, namely
viable_for_head_roots_and_weights
, so this PR leverages on the existingForkChoiceTestExecutor
implementation and extend it to adopt the suite. However, existingForkChoiceTestExecutor
implementation and the suite design details required a more extensive changes to the codebase, the following list might not be exhaustive:blsSetting
frommeta.yaml
and disable BLS for block and attestation processing if that is required by the setting, likewise, do fake pubkey aggregation computation in the sync committee updates during epoch transition. For performance reasons the new suite generates tests with fake BLS that should run with BLS disabled. The support is added via provisioningSpecConfig
with the BLS setting, this is done to avoid a ton of boilerplate that would be required in the next sync committee update case.valid
flag support for attestations is added.viable_for_head_roots_and_weights
check — this entailed implementation ofgetViableChainHeads
method.Known issues
A few tests (around 10) from the
tst_standard
corpus fail due to the following reasons:UNKNOWN_BLOCK
attestation statusCommittee information must be derived from a state no older than the previous epoch
The above looks like fair optimisation measure protecting from spamming nodes with the old attestations that have almost no value.
A couple of tests from the
tst_standard
corpus fail due to the following spec issue:Documentation
doc-change-required
label to this PR if updates are required.Changelog