Skip to content
This repository has been archived by the owner on Jun 11, 2024. It is now read-only.

Improve handling of deleted blocks #1896

Merged
merged 27 commits into from
Nov 10, 2023

Conversation

priojeetpriyom
Copy link
Contributor

@priojeetpriyom priojeetpriyom commented Oct 24, 2023

What was the problem?

This PR resolves #1191

How was it solved?

  • Pause indexing before block deletion and resume afterwards
  • Schedule block deletion first and index later on fork
  • Introduce backoff strategy for indexer jobs and increase attempts. This will help to reduce connector load when all calls are failing and reduce block deletion skip due because of failure to minimum.
  • Throw error on delete block message processor error. So that it is reprocessed

How was it tested?

  • Digital ocean droplets with pattern: priojeet-testnet-[1-4]-improve-handling-of-deleted-blocks

@priojeetpriyom priojeetpriyom self-assigned this Oct 24, 2023
@codecov
Copy link

codecov bot commented Oct 24, 2023

Codecov Report

Merging #1896 (ff7f435) into release/0.7.0 (20a0138) will increase coverage by 0.38%.
The diff coverage is 56.25%.

Impacted file tree graph

@@                Coverage Diff                @@
##           release/0.7.0    #1896      +/-   ##
=================================================
+ Coverage          66.62%   67.01%   +0.38%     
=================================================
  Files                298      298              
  Lines               5250     5254       +4     
  Branches             949      954       +5     
=================================================
+ Hits                3498     3521      +23     
+ Misses              1752     1733      -19     
Files Coverage Δ
services/blockchain-connector/config.js 100.00% <100.00%> (ø)
...-indexer/shared/dataService/business/generators.js 75.00% <ø> (+28.57%) ⬆️
...ndexer/shared/dataService/business/transactions.js 27.52% <ø> (ø)
services/blockchain-indexer/shared/regex.js 100.00% <100.00%> (ø)
...teway/apis/http-version3/methods/blockchainApps.js 100.00% <ø> (ø)
...is/http-version3/methods/blockchainAppsMetaList.js 100.00% <ø> (ø)
...is/http-version3/methods/blockchainAppsMetadata.js 100.00% <ø> (ø)
...tp-version3/methods/blockchainAppsTokenMetadata.js 100.00% <ø> (ø)
services/gateway/shared/regex.js 100.00% <100.00%> (ø)
services/gateway/tests/constants/paramValidator.js 100.00% <ø> (ø)
... and 2 more

... and 3 files with indirect coverage changes

services/blockchain-indexer/shared/dataService/blocks.js Outdated Show resolved Hide resolved
services/blockchain-indexer/config.js Outdated Show resolved Hide resolved
services/blockchain-indexer/config.js Outdated Show resolved Hide resolved
@priojeetpriyom priojeetpriyom force-pushed the 1191-improve-handling-of-deleted-blocks branch from 84a0fa1 to ee1d26a Compare November 9, 2023 13:46
@sameersubudhi sameersubudhi force-pushed the 1191-improve-handling-of-deleted-blocks branch from 04cf0a6 to afd8f40 Compare November 9, 2023 15:37
@sameersubudhi sameersubudhi force-pushed the 1191-improve-handling-of-deleted-blocks branch from afd8f40 to c2d0fdf Compare November 9, 2023 15:53
@sameersubudhi sameersubudhi force-pushed the 1191-improve-handling-of-deleted-blocks branch from 1d8594f to 004b8c6 Compare November 10, 2023 09:44
Copy link
Contributor

@nagdahimanshu nagdahimanshu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@nagdahimanshu nagdahimanshu force-pushed the 1191-improve-handling-of-deleted-blocks branch from 061fffe to 6b8dc0e Compare November 10, 2023 11:44
@nagdahimanshu nagdahimanshu force-pushed the 1191-improve-handling-of-deleted-blocks branch from 6b8dc0e to 19b8749 Compare November 10, 2023 11:48
@sameersubudhi sameersubudhi merged commit 73561e9 into release/0.7.0 Nov 10, 2023
6 of 8 checks passed
@sameersubudhi sameersubudhi deleted the 1191-improve-handling-of-deleted-blocks branch November 10, 2023 12:00
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants