fix contract type detection: get sighashes from all basicblocks

[mullakhmetov]

#353 #319 #349 #195 #194 and maybe more

What's done

1. Walk through all basicblocks to collect sighashes
2. Check all EIP20 methods signatures in is_erc721_contract
3. Add special check for CryptoKitties contract (and a way to add more ones)

@medvedev1088 I've noticed your concern about possible false positives in case of looking at all blocks in get_function_sighashes, so I've done some measurements.

Data

I walked through blocks in range [1_000_000, 16_000_000] in 100_000 blocks steps, scaned 1000 blocks in each step for contract calls and gathered all unique contract addresses. So I got ~1.3 contracts in total and picked 1k of them randomly.

For control type detection I used bitquery. Questionable, but it was the easiest way for me.

Results

version	accuracy
blockchain-etl:develop	81.2%
mullakhmetov:fix-contract-type-detection	95.1%

All the differences control vs. blockchain-etl:develop vs. mullakhmetov:fix-contract-type-detection:

contract address	control	blockchain-etl:develop	mullakhmetov:fix-contract-type-detection
0xd98f75b1a3261dab9eed4956c93f33749027a964	ERC20		ERC20
0x89fcb32f29e509cc42d0c8b6f058c993013a843f	ERC20		ERC20
0x1cd926f3e12f7b6c2833fbe7277ac53d529a794e	ERC20		ERC20
0x1cd926f3e12f7b6c2833fbe7277ac53d529a794e	ERC20		ERC20
0x1cd926f3e12f7b6c2833fbe7277ac53d529a794e	ERC20		ERC20
0x1cd926f3e12f7b6c2833fbe7277ac53d529a794e	ERC20		ERC20
0x1cd926f3e12f7b6c2833fbe7277ac53d529a794e	ERC20		ERC20
0x1cd926f3e12f7b6c2833fbe7277ac53d529a794e	ERC20		ERC20
0x1cd926f3e12f7b6c2833fbe7277ac53d529a794e	ERC20		ERC20
0x1cd926f3e12f7b6c2833fbe7277ac53d529a794e	ERC20		ERC20
0x16484d73ac08d2355f466d448d2b79d2039f6ebb	ERC20		ERC20
0x475c311491b92907272ffcf0bd77a286919ea98d	ERC20		ERC20
0xb268c1c44a349d06a42cf24988162dadc48d839e	ERC20		ERC20
0x5994b480f6e06d37a3df320cf4dc42c9cada5861	ERC20		ERC20
0x580b52748bbd472724458f97e4f0085286273911	ERC20
0x4ddc2d193948926d02f9b1fe9e1daa0718270ed5	ERC20		ERC20
0x00c8b2fd3213f089604e7a7ae264e9839a91f097	ERC20		ERC20
0x88a9a52f944315d5b4e917b9689e65445c401e83	ERC20		ERC20
0xf7421f5a5d4b9d0e93f81ec9dcbfd2fafb15e172	ERC20
0xf989e16233ad1ad0f2af739547ef92745341f52b	ERC20		ERC20
0x7f39c581f595b53c5cb19bd0b3f8da6c935e2ca0	ERC20		ERC20
0x584936357d68f5143f12e2e64f0089db93814dad	ERC20		ERC20
0xdc508199d2b0127da9c32c2829bdf880902de493	ERC20		ERC20
0x239362d589e641d818d62eb7dbc970e49e5ff592	ERC20		ERC20
0x77777777772cf0455fb38ee0e75f38034dfa50de	ERC20		ERC20
0xa58a4f5c4bb043d2cc1e170613b74e767c94189b	ERC20		ERC20
0xccf4429db6322d5c611ee964527d42e5d685dd6a	ERC20		ERC20
0x4b317864a05c91225ab8f401ec7be0aeb87e9c12	ERC20
0xf6865fcbe5c65c21148f3be37f71f16a55f29170	ERC20		ERC721
0x549020a9cb845220d66d3e9c6d9f9ef61c981102	ERC20		ERC20
0x40902c98b878471d1e50a846b016525bc8bef836	ERC20		ERC20
0xbb8be4772faa655c255309afc3c5207aa7b896fd			ERC20
0x83519ec12329fc296573b8e95e62f01313d63ca4	ERC20		ERC20
0x7870cc63b6b1af0aed0d6dd7c1efb39300b773eb	ERC20		ERC721
0x68e54af74b22acaccffa04ccaad13be16ed14eac	ERC20		ERC20
0x219b8ab790decc32444a6600971c7c3718252539	ERC721		ERC721
0xd8f2eefed38f10e9b53952125cfe66d6d1511559	ERC721		ERC721
0xf5cb0565704e821f0920b72d9a28a65bf327d901	ERC20		ERC20
0x7778360f035c589fce2f4ea5786cbd8b36e5396b	ERC20		ERC20
0x795dbf627484f8248d3d6c09c309825c1563e873	ERC20		ERC20
0x12d7d45a4b9693b312ede375074a48b9b9f2b6ec	ERC20		ERC20
0xc2fcab14ec1f2dfa82a23c639c4770345085a50f	ERC20		ERC20
0x1e4e46b7bf03ece908c88ff7cc4975560010893a	ERC20
0xad6917fc71101d377f5cbbc5b1fc0e941f08ae58	ERC20		ERC721
0xd1169e5349d1cb9941f3dcba135c8a4b9eacfdde	ERC721		ERC721
0xd1169e5349d1cb9941f3dcba135c8a4b9eacfdde	ERC721		ERC721
0xd1169e5349d1cb9941f3dcba135c8a4b9eacfdde	ERC721		ERC721
0xd1169e5349d1cb9941f3dcba135c8a4b9eacfdde	ERC721		ERC721
0xd1169e5349d1cb9941f3dcba135c8a4b9eacfdde	ERC721		ERC721
0xd1169e5349d1cb9941f3dcba135c8a4b9eacfdde	ERC721		ERC721
0xd1169e5349d1cb9941f3dcba135c8a4b9eacfdde	ERC721		ERC721
0xd1169e5349d1cb9941f3dcba135c8a4b9eacfdde	ERC721		ERC721
0xd184b10ee51be2c6bf45214b048f311a15296cae	ERC20
0x7d2d3688df45ce7c552e19c27e007673da9204b8	ERC20		ERC20
0x53109fe9e044f2c324d00ad85bfb0b13ce379480	ERC20		ERC20
0x00b784c0e9dd20fc865f89d05d0ce4417efb77a9	ERC721		ERC721
0xd06b25f67a17f12b41f615b34d87ecd716ff55a0	ERC20		ERC20
0x66a0f676479cee1d7373f3dc2e2952778bff5bd6	ERC20		ERC20
0xd5a73977e6effc252a7f1af4548ba11ff7187616	ERC20		ERC721
0x6a1fbe8f9e42fdf56519a0436f6a5f7ed509c7b1	ERC20		ERC721
0x8642a849d0dcb7a15a974794668adcfbe4794b56	ERC20		ERC20
0xb983499b7d2f2569a7399048bbd72ca65e4e44e2	ERC20		ERC20
0x29430893589f627046776e906efa281e410bef69			ERC20
0x4e2781e3ad94b2dfcf34c51de0d8e9358c69f296	ERC20		ERC721
0x0c4a68cf6857cc76fe946d04fe85fac5fae9625e	ERC20		ERC20
0xc32ab5f17c0edaeb4d597bfb9726f28bc37fd6af	ERC20		ERC20
0x2dec96736e7d24e382e25d386457f490ae64889e	ERC721		ERC721
0xfe5392049543e1fdcfad9cd8a05a6d28eef5e9b7	ERC20
0xa37702c29b2ba7c72d0bd3d5a0b6c9e7221dc84a	ERC721		ERC721
0xf9ab7776c7baeed1d65f0492fe2bb3951a1787ef	ERC20		ERC20
0xdb05ea0877a2622883941b939f0bb11d1ac7c400	ERC20		ERC20
0x3893b9422cd5d70a81edeffe3d5a1c6a978310bb	ERC20
0xc23b12eba3af92dc3ec94744c0c260cad0eed0e5	ERC721		ERC721
0x1104c2f267861aad82a3bad48ec4bf78ef221c63			ERC20
0xc6e64729931f60d2c8bc70a27d66d9e0c28d1bf9	ERC20		ERC20
0x8364d03349e76386263005ce79517a1776ddff45			ERC20
0xae06088ecb0e668db494ee64ae78af5e5b510797	ERC20		ERC20
0xf79a07cd3488bbafb86df1bad09a6168d935c017	ERC20		ERC20
0x7592dc0c4bc7cfc91ae5a7d62dc96c949fd27193	ERC20		ERC20
0xa467a1b040dba0f3e42c3f5208f8a3f826890047	ERC20		ERC721
0xbf7045f6ea651abb04e96cba61adabe6d7bf0ee8	ERC20		ERC20
0xa1faa113cbe53436df28ff0aee54275c13b40975	ERC20		ERC20
0x2d9fd51e896ff0352cb6d697d13d04c2cb85ca83	ERC20		ERC20
0x1877050a70a753aaac65743ff43199e08b6e2fb8	ERC20		ERC20
0x5b230ac0a27133ac14487a8c679172d1b90c18b0	ERC20		ERC20
0xaf5191b0de278c7286d6c7cc6ab6bb8a73ba2cd6	ERC20		ERC20
0xccc1825cf04cae4d497b202d1434ec0f79ee535f	ERC20
0x80c62fe4487e1351b47ba49809ebd60ed085bf52	ERC20		ERC20
0x9928e4046d7c6513326ccea028cd3e7a91c7590a	ERC20		ERC20
0xe72e2222100be1b61a8203fe1beec1f7f3fb4094	ERC721		ERC721
0x4eeea7b48b9c3ac8f70a9c932a8b1e8a5cb624c7	ERC20		ERC20
0xfe58fc763ba915013aee93c6f0890bfddbe03608	ERC721		ERC721
0x0a896423178a6699a975e98da5a14e4599d8019f	ERC721		ERC721
0x912656188616e0184e3181f019022990a63280b1	ERC20		ERC20
0x87c9524237a19338be7dbcac01d6d208ff31136f	ERC20		ERC20
0x763308a9bbfaa76790bfcbc9b160e8fa447b2d82	ERC20		ERC20
0xa3aee8bce55beea1951ef834b99f3ac60d1abeeb	ERC20		ERC721
0x755c1a8f71f4210cd7b60b9439451efcbeba33d1	ERC20		ERC20
0x15cc16bfe6fac624247490aa29b6d632be549f00	ERC721		ERC721
0xf1f3ca6268f330fda08418db12171c3173ee39c9	ERC20
0x8d41f6b4c017eaab099c0bae3ff49b4485539535	ERC20		ERC20
0x1985365e9f78359a9b6ad760e32412f4a445e862	ERC20
0x5cbefe14c66ba706e79ca4237ff10f218461014a	ERC20		ERC20
0x6f40d4a6237c257fff2db00fa0510deeecd303eb	ERC20
0x9d7107c8e30617cadc11f9692a19c82ae8bba938	ERC20		ERC20
0x53aabccae8c1713a6a150d9981d2ee867d0720e8	ERC20		ERC20
0x2e1c3b37e9c2e1ea1855b6b3fe886ae0f361757f	ERC20		ERC20
0x63ec6726d5e4615cfcc75988959c36798177fe60	ERC20		ERC20
0x7b50775383d3d6f0215a8f290f2c9e2eebbeceb2	ERC20		ERC20
0xa1849880593e96d2f7df77d0d38a7f2372ae10e0			ERC20
0x5ae854b098727a9f1603a1e21c50d52dc834d846			ERC20
0x0aa7efe4945db24d95ca6e117bba65ed326e291a	ERC20		ERC20
0x8bf2f876e2dcd2cae9c3d272f325776c82da366d	ERC721		ERC721
0xf1d245382862bbd5a32384e66bf8b59c4b09de2f	ERC20		ERC20
0x61247d8aca1c485a50728e1336d9b26c8339e701	ERC20		ERC20
0x76306f029f8f99effe509534037ba7030999e3cf	ERC20		ERC20
0xc4d9102e36c5063b98010a03c1f7c8bd44c32a00	ERC20		ERC20
0x797e1dc0738f2f047d7e0dd185e41a35f87c5618	ERC20		ERC20
0x0968c90198f08b67365840fa37631b29fe2aa9fc	ERC20		ERC20
0x35242ed010d80f39643982d0216d7ca7ad1fa841	ERC20		ERC20
0x18aaa7115705e8be94bffebde57af9bfc265b998	ERC20
0x793e2602a8396468f3ce6e34c1b6c6fd6d985bad	ERC20		ERC20
0x10b66bd1e3b5a936b7f8dbc5976004311037cdf0	ERC721		ERC721
0xc98d64da73a6616c42117b582e832812e7b8d57f	ERC20		ERC20
0x06677dc4fe12d3ba3c7ccfd0df8cd45e4d4095bf	ERC20		ERC20
0x425e0ec09f9bcdfd20283bf23251ccb74d118813			ERC20
0x208bd5dc470eba21571ddb439801a614ed346376	ERC20		ERC20
0xefaa7f81c5397a73b43f340fe3397281c07a4057	ERC20
0xfcbb61bcd4909bf4af708f15aaaa905e0978cafc	ERC20
0xb1bd9d7942a250ba2dce27dd601f2ed4211a60c4	ERC721		ERC721
0xa2b4c0af19cc16a6cfacce81f192b024d625817d	ERC20		ERC20
0xffc7f8a6fcd1d01b295514144eae6e781a2cdf15	ERC721
0xeb7f9b192c7167ac3aeca396503732023aa1326a	ERC20		ERC20
0xc90b47731f9cd4db0df36c1e59fdca1fe9806475			ERC20
0x4028daac072e492d34a3afdbef0ba7e35d8b55c4	ERC20		ERC20
0xec86e12a083623c6dd66ebd594d99218449c73aa	ERC20		ERC20
0x47f3a38990ca12e39255e959f7d97fbe5906afd4	ERC721		ERC721
0xeef9f339514298c6a857efcfc1a762af84438dee	ERC20		ERC20
0x971e22a354fe026faf437dd283a21fd5bf57b4fa	ERC20		ERC20
0x8e0cc15bbcd10e170ac07982b5d6930502c63784		ERC20	ERC20
0xf650c3d88d12db855b8bf7d11be6c55a4e07dcc9	ERC20		ERC20
0x849d92c7a869160bc9bba3b9e7b2d052bc094a2c	ERC20
0xc18360217d8f7ab5e7c516566761ea12ce7f9d72	ERC20		ERC20
0x9ff73c7adb72af71e52b55d27e269fe587ad9c6c	ERC20		ERC20
0x3459de17141dee94b12ee5816ff9650c0a99b371			ERC20
0xae461ca67b15dc8dc81ce7615e0320da1a9ab8d5	ERC20		ERC20
0xc9ca10d36441b5b45d5e63480139105f037972e0	ERC20		ERC20
0x7703c35cffdc5cda8d27aa3df2f9ba6964544b6e	ERC20
0xd0d932c9f78583d297e487a7965223d0fe1008c8	ERC20
0x22674fd4ce74765c211eec01698fda36f57a650a	ERC721		ERC721
0xca7c2771d248dcbe09eabe0ce57a62e18da178c0	ERC20		ERC20
0x04f2694c8fcee23e8fd0dfea1d4f5bb8c352111f	ERC20		ERC20
0x633cfa9ac099d18c1b33736892427f0a7c1d120a	ERC20		ERC721
0xd03dffd02db4b076e3eed9272a6a243a23de16e3	ERC20		ERC20
0x7dd3f5705504002dc946aeafe6629b9481b72272	ERC20		ERC20
0xc8645f0d6a622e2a9a66c1db9d33b1a46b4462dc	ERC20		ERC20
0x2d9e5de7d36f3830c010a28b29b3bdf5ca73198e	ERC20	ERC721	ERC721
0xb4d06d46a8285f4ec79fd294f78a881799d8ced9	ERC721		ERC721
0xc770eefad204b5180df6a14ee197d99d808ee52d	ERC20		ERC20
0xecba967d84fcf0405f6b32bc45f4d36bfdbb2e81	ERC20		ERC20
0xfee4800067bfc9dff564d116cba4d4b16ca7b7b3	ERC20		ERC20
0xa7ad4ce6c21d9e875f1067cd377256326b483002	ERC20		ERC20
0x17fbba7388302f835ba2ded8d18b5248d0ad3d45	ERC20		ERC20
0xe0fa9fb0e30ca86513642112bee1cbbaa2a0580d	ERC20		ERC721
0x348b7f3106b5da47405332534d06069ff9ce4d1b	ERC20		ERC20
0x74876d936997684a38a2aa6ed5aa25de5f09ff47	ERC721
0x38fb3b5de4ba84f6ac82be6eaffb8f21e75516a4	ERC20
0xcbfef8fdd706cde6f208460f2bf39aa9c785f05d	ERC20		ERC20
0x88acdd2a6425c3faae4bc9650fd7e27e0bebb7ab	ERC20		ERC20
0xdf574c24545e5ffecb9a659c229253d4111d87e1	ERC20		ERC20
0x1ffe8a8177d3c261600a8bd8080d424d64b7fbc2	ERC20		ERC20
0x34c080e1ea5a5c98301bd185dde2dca150e96a60	ERC20		ERC20
0x490ccb3c835597ff31e525262235487f9426312b	ERC20		ERC20
0xd1e5b0ff1287aa9f9a268759062e4ab08b9dacbe	ERC721		ERC721
0x2157a7894439191e520825fe9399ab8655e0f708	ERC20		ERC20
0x4fb721ef3bf99e0f2c193847afa296b9257d3c30	ERC20		ERC20
0x04fa0d235c4abf4bcf4787af4cf447de572ef828	ERC20		ERC20
0x0c1b3468285d092d40d0453a05f3ea87f1ed120b			ERC721
0x2e81ec0b8b4022fac83a21b2f2b4b8f5ed744d70	ERC20		ERC20
0x6a091a3406e0073c3cd6340122143009adac0eda	ERC20		ERC20
0x3a3a65aab0dd2a17e3f1947ba16138cd37d08c04	ERC20		ERC20
0xde84d2ecacff65a6789cd4480a0ce8230006fad4	ERC721		ERC721
0x26ea744e5b887e5205727f55dfbe8685e3b21951	ERC20		ERC20
0xe56c60b5f9f7b5fc70de0eb79c6ee7d00efa2625	ERC20		ERC20
0x6afb3d434ac3d0c92e5397895f17f1f9af22a6c2	ERC20		ERC20
0xa2107fa5b38d9bbd2c461d6edf11b11a50f6b974	ERC20		ERC20
0xb982fad15538647743b983f4aa5deaf8266e0d67	ERC20		ERC20
0x0417912b3a7af768051765040a55bb0925d4ddcf	ERC20
0x3b4f7cb9e60362a49dd04eb0091a374d340e3efd	ERC20		ERC20
0xd1146680ae5a4d71f68accb6610030ced7e7afe1	ERC20		ERC721
0x31503dcb60119a812fee820bb7042752019f2355	ERC20		ERC20
0xa06fda2caa66148603314451ba0f30c9c5d539e3	ERC20		ERC721
0xab7fa2b2985bccfc13c6d86b1d5a17486ab1e04c	ERC20
0xec213f83defb583af3a000b1c0ada660b1902a0f	ERC20
0xfd51aa745a5803db66de8f853765e83981a83bf4			ERC20
0x5f3b5dfeb7b28cdbd7faba78963ee202a494e2a2	ERC20
0xa0f4563fc991269f5a90250eb44c583a943df3b1	ERC20		ERC20
0x9a4d5580497abde06df106b5cd2d2ecbed7b09c9	ERC20		ERC20
0x06ff1a3b08b63e3b2f98a5124bfc22dc0ae654d3	ERC20		ERC20

[FahdW]

I am curious, how many derivations of the crypto kitty contract is there? Is there not just one official copy of it? Wouldn't it make more sense to just hardcode that contract instead of running a check each time something runs though the is_crypto_kitty function