Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Pub/Sub attribute size exceeded #472

Open
diega opened this issue Jan 23, 2024 · 0 comments
Open

Google Pub/Sub attribute size exceeded #472

diega opened this issue Jan 23, 2024 · 0 comments

Comments

@diega
Copy link

diega commented Jan 23, 2024

I'm attempting to stream Ethereum Classic traces to Google Pub/Sub using v2.3.1, but when the process reaches block 1431916 (before the DAO fork, i.e. existing in ETH too) I get the following error:

google.api_core.exceptions.InvalidArgument: 400 The attribute "item_id" in the request has a value that is too long. The length is 1993 characters, but the maximum allowed is 1024. Refer to https://cloud.google.com/pubsub/quotas for more information.

I'm able to reproduce the behavior using this set of options

stream
--provider-uri=http://localhost:8545
-e trace
--batch-size 1
--block-batch-size 1
--output projects/<your_project>/topics/<your_topic>
--start-block 1431916

Following the documentation quoted in the error, it states that the maximum size for any attribute is 1024 bytes so stringifying the traceAddress to set the trace_id exceeds the limit.

For the record, here's the json response to the trace_block call.

If you query the public Big Query tables from Google, you can see that these traces are properly stored, so I suspect they are not filling the tables using Pub/Sub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant