Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a binary to extract data from fuel-core #8

Merged
merged 3 commits into from
Feb 27, 2024

Conversation

Dentosal
Copy link
Member

@Dentosal Dentosal commented Feb 22, 2024

Migrated from fuel-core, as this is fully indepdendent and only using the GraphQL API. Should be compatible with fuel-core versions ranging from 0.21.0 used by Beta-5 network, to the latest master at FuelLabs/fuel-core@ed92738

I'm not sure if I'm supposed to regenerate the go files here, or do we do that in a separate PR?

Nice to have things, possible follow-up PRs:

  • More options for e.g. pulling old data
  • Better error handling, e.g. attempting to reconnect on network errors
  • Automated tests
  • CI checks

The crate is currently missing license field on Cargo.toml. Is it ok to license this under Apache-2, or do we need BSL here @xgreenx?


let mut tx_data: Vec<Transaction> = vec![];
let mut tx_extra: Vec<TxExtra> = vec![];
for id in &block.transactions {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting all transactions one by one will be very slow and expensive=)

It is better to request all information in one request. You can check how it was done on the indexer side https://github.com/FuelLabs/fuel-indexer/pull/969/files

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postponed to #11

let mut height: BlockHeight = opt.height.into();

loop {
if let Some(block) = client.block_by_height(height.into()).await? {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fuel-core support PaginatedResult, so it would be better to request a lot of blocks at the same time

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We mainly poll for the latest block, but it still probably makes sense for pulling the old blocks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postponed to #11

@Salka1988 Salka1988 merged commit 67e9597 into master Feb 27, 2024
1 check passed
@Salka1988 Salka1988 deleted the dento/extract-from-fuel-core branch February 27, 2024 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants