feat(indexer): add to modules and implement proto fields (backport #22544) #22676
+549
−159
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
(cherry picked from commit bd76b47) # Conflicts: # collections/CHANGELOG.md # collections/go.mod # collections/go.sum # collections/indexing.go # collections/protocodec/collections.go # collections/protocodec/collections_test.go # indexer/postgres/create_table.go # indexer/postgres/enum.go # indexer/postgres/listener.go # indexer/postgres/tests/testdata/init_schema.txt # indexer/postgres/tests/testdata/init_schema_no_retain_delete.txt # math/legacy_dec_test.go # schema/appdata/async.go # schema/field.go # schema/field_test.go # schema/state_object.go # schema/state_object_test.go # server/v2/cometbft/go.mod # server/v2/cometbft/go.sum # simapp/go.mod # simapp/go.sum # store/cachekv/branch_bench_test.go # tests/integration/v2/auth/app_test.go # tests/integration/v2/services.go # x/bank/keeper/view.go # x/circuit/go.mod # x/circuit/go.sum # x/consensus/go.mod # x/consensus/go.sum # x/distribution/go.mod # x/epochs/go.mod # x/epochs/go.sum # x/evidence/go.mod # x/evidence/go.sum # x/mint/go.mod # x/mint/go.sum # x/nft/go.mod # x/nft/go.sum # x/params/go.mod # x/params/go.sum # x/protocolpool/go.mod # x/protocolpool/go.sum # x/slashing/go.mod # x/slashing/go.sum # x/staking/go.mod # x/upgrade/go.mod # x/upgrade/go.sum
|
Cherry-pick of bd76b47 has failed: To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally |
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
github-actions
bot
added
C:x/staking
C:CLI
C:x/gov
C:x/slashing
C:x/distribution
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Comment on lines
903
to
+911
| defer func() { | ||
| // call the streaming service hooks with the FinalizeBlock messages | ||
| for _, streamingListener := range app.streamingManager.ABCIListeners { | ||
| if err := streamingListener.ListenFinalizeBlock(app.finalizeBlockState.Context(), *req, *res); err != nil { | ||
| if streamErr := streamingListener.ListenFinalizeBlock(app.finalizeBlockState.Context(), *req, *res); streamErr != nil { | ||
| app.logger.Error("ListenFinalizeBlock listening hook failed", "height", req.Height, "err", err) | ||
| if app.streamingManager.StopNodeOnErr { | ||
| // if StopNodeOnErr is set, we should return the streamErr in order to stop the node | ||
| err = streamErr | ||
| } |
There was a problem hiding this comment.
Change potentially affects state.
Call sequence:
(*github.com/cosmos/cosmos-sdk/baseapp.BaseApp).FinalizeBlock (baseapp/abci.go:902)
|
DNM, will fix replaces in main |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
#22677 should be cherry-picked in here too before the conflict resolution |
|
@facundomedica your pull request is missing a changelog! |
julienrbrt
reviewed
Nov 28, 2024
julienrbrt
approved these changes
Nov 28, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Implement necessary methods and named keys in:
Changed some default key codecs to be
NameableKeyCodec(TimeKey, LengthPrefixedBytesKey, LengthPrefixedAddressKey)Improve error messages in schema
Fixed AsyncListener in schema to avoid hanging on unprocessed packets
In baseapp, check for
app.streamingManager.StopNodeOnErrso the indexer can crash the node if needed. Also the call to the store's Commit() was moved to after the streaming listener, so in case that something fails in the indexer we can stop the node before it commits. Otherwise when restarting the indexer would have skipped a block.In codec:
The following is a bit hacky:
In collections, for ModuleCodec, we use the value decoder to obtain an empty message (so this assumes that value decoders will return an empty struct and not a nil when passing an empty byte slice). This is used to optionally get a proto message which is then used to register any existing enums in the used messages.
We also adapt the full enum name to not use periods, otherwise it will fail somewhere else (
Name: strings.ReplaceAll(string(enum.FullName()), ".", "_"), // make it compatible with schema).The goal of this is so then the indexer can register the enum in the DB, which need to be registered before starting to write data.
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
included the correct type prefix in the PR title, you can find examples of the prefixes below:
confirmed
!in the type prefix if API or client breaking changetargeted the correct branch (see PR Targeting)
provided a link to the relevant issue or specification
reviewed "Files changed" and left comments if necessary
included the necessary unit and integration tests
added a changelog entry to
CHANGELOG.mdupdated the relevant documentation or specification, including comments for documenting Go code
confirmed all CI checks have passed
Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.
I have...
Summary by CodeRabbit
Release Notes
New Features
ModuleCodecmethods in multiple modules to support schema management and indexing.collectionsmodule.RouterServicefor structured message handling in integration tests.Bug Fixes
Dependencies
Chores
This is an automatic backport of pull request #22544 done by [Mergify](https://mergify.com).