Optimize ERC20 retrieval by using Filters

[medvedev1088]

Using log filters instead of retrieving transaction receipts significantly improves performance of ERC20 retrieval https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_getfilterlogs

My tests show 100x improvement for some blocks. Here is an example implementation in python https://github.com/medvedev1088/ethereum-etl/blob/88fd6792707fa8c0fdfd5ff199bc3ff44b9a1aae/ethereumetl/jobs.py#L155

[xiaoyao1991]

Good point. I didn't choose to implement that way because Infura does not support this API call yet.

[medvedev1088]

What do you think about ERC20RetrievalStrategy with 2 implementations ReceiptRetrievalStrategy, LogFilterRetrievalStrategy? It can be configured with a property, erc20.retrieval-strategy=receipt|log-filter|auto.

BTW, do you know if Infura have plans to add support for filter APIs?

[xiaoyao1991]

I like that.

I don't know if Infura will support filter APIs. It's kinda hard cuz having filters makes their API stateful.