You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am using ethereum-etl to parse all specific token transfers from start to end block. As a node I'm running a local snap node with all the receipts being saved parameter (--txlookuplimit 0).
To test this instruments work I've decided to parse USDT token (0xdAC17F958D2ee523a2206206994597C13D831ec7) transfers from 20_500_000 block to 20_505_000 block.
The issue I'am having that it works really slow. I am talking taking about 15±0.5 seconds to parse all the transfers. The official docs state that:
You can tune --batch-size, --max-workers for performance.
But tunning --max-workers parameters actually gives me a slight decrease in speed; --batch-size set to 4096 increases speed to 12.4±0.3 seconds, but it's further tweaking worsens the result.
For my original task I woult need to parse millisons of blocks, so the current speed is unacceptable.
I am using ethereum-etl to parse all specific token transfers from start to end block. As a node I'm running a local snap node with all the receipts being saved parameter (
--txlookuplimit 0
).To test this instruments work I've decided to parse USDT token (
0xdAC17F958D2ee523a2206206994597C13D831ec7
) transfers from20_500_000
block to20_505_000
block.The issue I'am having that it works really slow. I am talking taking about 15±0.5 seconds to parse all the transfers.
The official docs state that:
But tunning
--max-workers
parameters actually gives me a slight decrease in speed;--batch-size
set to4096
increases speed to 12.4±0.3 seconds, but it's further tweaking worsens the result.For my original task I woult need to parse millisons of blocks, so the current speed is unacceptable.
The command I'm running:
I've read this Medium article: How to Export the Entire Ethereum Blockchain to CSV in 2 hours for $10, but I can replicate, since I can't get access to AWS services.
Am I facing a hardware bottleneck or is there something I can do to make it faster?
Would much appreciate any suggestions!
The text was updated successfully, but these errors were encountered: