somethinig wrong with ethereumctl stream

[AutuBread]

`-> % ethereumetl stream --start-block 500000 --provider-uri https://rpc.ankr.com/eth --output=postgresql+pg8000://postgres:[email protected]:5432/postgres
2024-03-17 18:32:16,869 - root [INFO] - Using https://rpc.ankr.com/eth
2024-03-17 18:32:17,703 - root [INFO] - Current block 19453993, target block 500000, last synced block 499999, blocks to sync 1
2024-03-17 18:32:17,703 - ProgressLogger [INFO] - Started work. Items to process: 1.
2024-03-17 18:32:17,840 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2024-03-17 18:32:17,840 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.137233.
2024-03-17 18:32:17,841 - ProgressLogger [INFO] - Started work.
2024-03-17 18:32:17,841 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000199.
2024-03-17 18:32:17,841 - ProgressLogger [INFO] - Started work.
2024-03-17 18:32:17,842 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000169.
2024-03-17 18:32:17,842 - root [INFO] - Exporting with MultiItemExporter
2024-03-17 18:32:17,848 INFO sqlalchemy.engine.Engine select version()
2024-03-17 18:32:17,848 - sqlalchemy.engine.Engine [INFO] - select version()
2024-03-17 18:32:17,848 INFO sqlalchemy.engine.Engine [raw sql] ()
2024-03-17 18:32:17,848 - sqlalchemy.engine.Engine [INFO] - [raw sql] ()
2024-03-17 18:32:17,851 INFO sqlalchemy.engine.Engine select current_schema()
2024-03-17 18:32:17,851 - sqlalchemy.engine.Engine [INFO] - select current_schema()
2024-03-17 18:32:17,851 INFO sqlalchemy.engine.Engine [raw sql] ()
2024-03-17 18:32:17,851 - sqlalchemy.engine.Engine [INFO] - [raw sql] ()
2024-03-17 18:32:17,852 INFO sqlalchemy.engine.Engine show standard_conforming_strings
2024-03-17 18:32:17,852 - sqlalchemy.engine.Engine [INFO] - show standard_conforming_strings
2024-03-17 18:32:17,852 INFO sqlalchemy.engine.Engine [raw sql] ()
2024-03-17 18:32:17,852 - sqlalchemy.engine.Engine [INFO] - [raw sql] ()
2024-03-17 18:32:17,857 INFO sqlalchemy.engine.Engine INSERT INTO blocks (timestamp, number, hash, parent_hash, nonce, sha3_uncles, logs_bloom, transactions_root, state_root, receipts_root, miner, difficulty, total_difficulty, size, extra_data, gas_limit, gas_used, transaction_count, base_fee_per_gas) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) ON CONFLICT (hash) DO UPDATE SET timestamp = excluded.timestamp, number = excluded.number, parent_hash = excluded.parent_hash, nonce = excluded.nonce, sha3_uncles = excluded.sha3_uncles, logs_bloom = excluded.logs_bloom, transactions_root = excluded.transactions_root, state_root = excluded.state_root, receipts_root = excluded.receipts_root, miner = excluded.miner, difficulty = excluded.difficulty, total_difficulty = excluded.total_difficulty, size = excluded.size, extra_data = excluded.extra_data, gas_limit = excluded.gas_limit, gas_used = excluded.gas_used, transaction_count = excluded.transaction_count, base_fee_per_gas = excluded.base_fee_per_gas
2024-03-17 18:32:17,857 - sqlalchemy.engine.Engine [INFO] - INSERT INTO blocks (timestamp, number, hash, parent_hash, nonce, sha3_uncles, logs_bloom, transactions_root, state_root, receipts_root, miner, difficulty, total_difficulty, size, extra_data, gas_limit, gas_used, transaction_count, base_fee_per_gas) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) ON CONFLICT (hash) DO UPDATE SET timestamp = excluded.timestamp, number = excluded.number, parent_hash = excluded.parent_hash, nonce = excluded.nonce, sha3_uncles = excluded.sha3_uncles, logs_bloom = excluded.logs_bloom, transactions_root = excluded.transactions_root, state_root = excluded.state_root, receipts_root = excluded.receipts_root, miner = excluded.miner, difficulty = excluded.difficulty, total_difficulty = excluded.total_difficulty, size = excluded.size, extra_data = excluded.extra_data, gas_limit = excluded.gas_limit, gas_used = excluded.gas_used, transaction_count = excluded.transaction_count, base_fee_per_gas = excluded.base_fee_per_gas
2024-03-17 18:32:17,858 INFO sqlalchemy.engine.Engine [no key 0.00060s] ('2015-11-06 18:01:05', Decimal('500000'), '0xac8e95f7483f7131261bcc0a70873f8236c27444c940defc677f74f281220193', '0x794a1bef434928ce3aadd2f5eced2bf72ac714a30e9e4ab5965d7d9760300d84', '0x8901a71893cb3488', '0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347', '0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 ... (216 characters truncated) ... 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', '0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421', '0xb2bcfa2ffe869085c84a976435f1581a7a0eb7af64bafcbbda710661016aa3ab', '0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421', '0x580992b51e3925e23280efb93d3047c82f17e038', Decimal('7545189127997'), Decimal('2693928334782753024'), Decimal('541'), '0xd783010203844765746887676f312e352e31856c696e7578', Decimal('3141592'), Decimal('0'), Decimal('0'), None)
2024-03-17 18:32:17,858 - sqlalchemy.engine.Engine [INFO] - [no key 0.00060s] ('2015-11-06 18:01:05', Decimal('500000'), '0xac8e95f7483f7131261bcc0a70873f8236c27444c940defc677f74f281220193', '0x794a1bef434928ce3aadd2f5eced2bf72ac714a30e9e4ab5965d7d9760300d84', '0x8901a71893cb3488', '0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347', '0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 ... (216 characters truncated) ... 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', '0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421', '0xb2bcfa2ffe869085c84a976435f1581a7a0eb7af64bafcbbda710661016aa3ab', '0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421', '0x580992b51e3925e23280efb93d3047c82f17e038', Decimal('7545189127997'), Decimal('2693928334782753024'), Decimal('541'), '0xd783010203844765746887676f312e352e31856c696e7578', Decimal('3141592'), Decimal('0'), Decimal('0'), None)
2024-03-17 18:32:17,860 INFO sqlalchemy.engine.Engine ROLLBACK
2024-03-17 18:32:17,860 - sqlalchemy.engine.Engine [INFO] - ROLLBACK
2024-03-17 18:32:17,861 - root [ERROR] - An exception occurred while syncing block data.
Traceback (most recent call last):
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/pg8000/legacy.py", line 253, in execute
statement, vals=vals, oids=self._input_oids, stream=stream
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/pg8000/core.py", line 641, in execute_unnamed
self.handle_messages(context)
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/pg8000/core.py", line 778, in handle_messages
raise self.error
pg8000.exceptions.DatabaseError: {'S': 'ERROR', 'V': 'ERROR', 'C': '42P01', 'M': 'relation "blocks" does not exist', 'P': '13', 'F': 'parse_relation.c', 'L': '1180', 'R': 'parserOpenTable'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1706, in _execute_context
cursor, statement, parameters, context
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 681, in do_execute
cursor.execute(statement, parameters)
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/pg8000/legacy.py", line 279, in execute
raise cls(msg)
pg8000.dbapi.ProgrammingError: {'S': 'ERROR', 'V': 'ERROR', 'C': '42P01', 'M': 'relation "blocks" does not exist', 'P': '13', 'F': 'parse_relation.c', 'L': '1180', 'R': 'parserOpenTable'}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/blockchainetl/streaming/streamer.py", line 77, in _do_stream
synced_blocks = self._sync_cycle()
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/blockchainetl/streaming/streamer.py", line 98, in _sync_cycle
self.blockchain_streamer_adapter.export_all(self.last_synced_block + 1, target_block)
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/ethereumetl/streaming/eth_streamer_adapter.py", line 103, in export_all
self.item_exporter.export_items(all_items)
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/blockchainetl/jobs/exporters/multi_item_exporter.py", line 34, in export_items
exporter.export_items(items)
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/blockchainetl/jobs/exporters/postgres_item_exporter.py", line 51, in export_items
connection.execute(insert_stmt, converted_items)
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 314, in _execute_on_connection
self, multiparams, params, execution_options
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1399, in _execute_clauseelement
cache_hit=cache_hit,
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1749, in execute_context
e, statement, parameters, cursor, context
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1930, in handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from=e
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 198, in raise
raise exception
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1706, in _execute_context
cursor, statement, parameters, context
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 681, in do_execute
cursor.execute(statement, parameters)
File "/mnt/home/nyd/.local/lib/python3.6/site-packages/pg8000/legacy.py", line 279, in execute
raise cls(msg)
sqlalchemy.exc.ProgrammingError: (pg8000.dbapi.ProgrammingError) {'S': 'ERROR', 'V': 'ERROR', 'C': '42P01', 'M': 'relation "blocks" does not exist', 'P': '13', 'F': 'parse_relation.c', 'L': '1180', 'R': 'parserOpenTable'}
[SQL: INSERT INTO blocks (timestamp, number, hash, parent_hash, nonce, sha3_uncles, logs_bloom, transactions_root, state_root, receipts_root, miner, difficulty, total_difficulty, size, extra_data, gas_limit, gas_used, transaction_count, base_fee_per_gas) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) ON CONFLICT (hash) DO UPDATE SET timestamp = excluded.timestamp, number = excluded.number, parent_hash = excluded.parent_hash, nonce = excluded.nonce, sha3_uncles = excluded.sha3_uncles, logs_bloom = excluded.logs_bloom, transactions_root = excluded.transactions_root, state_root = excluded.state_root, receipts_root = excluded.receipts_root, miner = excluded.miner, difficulty = excluded.difficulty, total_difficulty = excluded.total_difficulty, size = excluded.size, extra_data = excluded.extra_data, gas_limit = excluded.gas_limit, gas_used = excluded.gas_used, transaction_count = excluded.transaction_count, base_fee_per_gas = excluded.base_fee_per_gas]
[parameters: ('2015-11-06 18:01:05', Decimal('500000'), '0xac8e95f7483f7131261bcc0a70873f8236c27444c940defc677f74f281220193', '0x794a1bef434928ce3aadd2f5eced2bf72ac714a30e9e4ab5965d7d9760300d84', '0x8901a71893cb3488', '0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347', '0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 ... (216 characters truncated) ... 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', '0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421', '0xb2bcfa2ffe869085c84a976435f1581a7a0eb7af64bafcbbda710661016aa3ab', '0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421', '0x580992b51e3925e23280efb93d3047c82f17e038', Decimal('7545189127997'), Decimal('2693928334782753024'), Decimal('541'), '0xd783010203844765746887676f312e352e31856c696e7578', Decimal('3141592'), Decimal('0'), Decimal('0'), None)]
(Background on this error at: http://sqlalche.me/e/14/f405)
2024-03-17 18:32:17,866 - root [INFO] - Nothing to sync. Sleeping for 10 seconds...
^Z
[8] + 6557 suspended ethereumetl stream --start-block 500000 --provider-uri `

[B1Q]

same issue unfortunately and no solution yet, i will let you know.

[AutuBread]

same issue unfortunately and no solution yet, i will let you know.

Actually, I solved the problem after manually establishing the relation in progresql......

[B1Q]

same issue unfortunately and no solution yet, i will let you know.

Actually, I solved the problem after manually establishing the relation in progresql......

I misread your error, mine was related to the transactions table column Value was not able to hold a specific value from a transaction and caused syncing to fail so i had to set value to Number(100, 70) for it to work which will be slow and use more disk space