Use organization package rpc-handler to get the fastest provider available

[gentlementlegen]

I had no luck when trying to use the rpc package so I want to do it in a separate PR. This endpoint was quite reliable during my tests which is why I kept it. You can even try it in my testing repo if you want.

Originally posted by @gentlementlegen in #2 (comment)

Depends on ubiquity/rpc-handler#45 to be published.

[gentlementlegen]

The following issue would need to be resolved to avoid the bugs that I encountered: ubiquity/rpc-handler#32

[ubiquibot]

@gentlementlegen the deadline is at 2024-08-08T04:27:09.489Z

[gentlementlegen]

The main issue currently is that the rpc-handler package makes too many requests and triggers a cancellation on the worker. There should be an option to limit the amount of tested networks in the package.

[gentlementlegen]

Update: I've been working all day on this, by changing the rpc-handler package to either include only specified URLs or even hardcoding a list of URLs. Every time I get the following message:

"HttpError: Too many subrequests."

Maybe our package is not suited for this. In the end we do not care about having the fastest RPC but any that works. Maybe it would be best to simply use the same package we use in rpc-handler, and try them one by one until the first one that works to avoid bursting the request limit within the workers. @rndquu thoughts?

[rndquu]

Update: I've been working all day on this, by changing the rpc-handler package to either include only specified URLs or even hardcoding a list of URLs. Every time I get the following message:

"HttpError: Too many subrequests."

Maybe our package is not suited for this. In the end we do not care about having the fastest RPC but any that works. Maybe it would be best to simply use the same package we use in rpc-handler, and try them one by one until the first one that works to avoid bursting the request limit within the workers. @rndquu thoughts?

You mean the HttpError: Too many subrequests error is thrown by Cloudflare while using the rpc-handler package, right?

[gentlementlegen]

@rndquu That is correct. The count for total request doesn't care if the request got aborted or cancelled it seems, so we always reach the limit by using out package.

[rndquu]

@gentlementlegen @Keyrxng

So the issue is that https://github.com/ubiquity/rpc-handler can't be used in a cloudflare environment (at least for now) because of a great number of HTTP requests in the background.

Perhaps we should add a new method to https://github.com/ubiquity/rpc-handler like handler.getAnyRpcProvider() which would simply return the 1st item from https://chainlist.org/chain/1 without any background checks?

Update: I've just double checked and there's already an issue for that ubiquity/rpc-handler#43 so it makes sense to put current issue on hold and implement ubiquity/rpc-handler#43 first.

[Keyrxng]

Yeah I think we could create a generator function that tests the network RPCs one by one and we yield the first to pass with checks if possible or none if we still hit that error

@gentlementlegen have you tried passing a hex network id and manually assigning network and runtime rpcs? Hacky and might not work but I'm sure the Networkish type also includes hex but rpc-handler wouldn't find a list of networkd RPCs.

I quickly read through the code and I think that you always end up with all of the endpoints instead of just the custom ones you passed?

by changing the rpc-handler package to either include only specified URLs or even hardcoding a list of URLs.

Oh I think you meant rewriting it, my mistake. Although I think the above still applies but what might stop us is if the random network RPC that we select is a slow/faulty one then it'll likely hit the max request time. So like @rndquu suggests just serving a url from the package without checks but that's the same as hardcoding 3 or 4 known good ones and shuffling isn't it?

[gentlementlegen]

I was trying to solve the issue on the rpc package by skipping all the other endpoints but I didn't expect them to still count as network calls, even canceled or failed. A method to try them one by one seems to be a reliable idea, I will look into that.

[ubiquibot]

@gentlementlegen the deadline is at 2024-08-18T14:08:53.387Z

[ubiquity-os]

[ 80.92 WXDAI ]

@gentlementlegen

Contributions Overview

View	Contribution	Count	Reward
Issue	Task	1	25
Issue	Specification	1	17.7
Issue	Comment	5	38.22
Review	Comment	2	0

Conversation Incentives

Comment	Formatting	Relevance	Reward
I had no luck when trying to use the rpc package so I want to do…	17.7 content: p: count: 59 score: 1 em: count: 6 score: 0 wordValue: 0.1 formattingMultiplier: 3	1	17.7
The following issue would need to be resolved to avoid the bugs …	3.2 content: p: count: 16 score: 1 wordValue: 0.2 formattingMultiplier: 1	0.6	1.92
The main issue currently is that the `rpc-handler` packa…	7.2 content: p: count: 35 score: 1 code: count: 1 score: 1 wordValue: 0.2 formattingMultiplier: 1	0.8	5.76
Update: I've been working all day on this, by changing the `…	20.2 content: p: count: 95 score: 1 code: count: 6 score: 1 pre: count: 4 score: 0 wordValue: 0.2 formattingMultiplier: 1	0.9	18.18
@rndquu That is correct. The count for total request doesn't car…	6 content: p: count: 30 score: 1 wordValue: 0.2 formattingMultiplier: 1	0.7	4.2
I was trying to solve the issue on the rpc package by skipping a…	10.2 content: p: count: 51 score: 1 wordValue: 0.2 formattingMultiplier: 1	0.8	8.16
Resolves #3 Depends on https://github.com/ubiquity/rpc-handler…	0 content: p: count: 6 score: 1 ul: count: 18 score: 0 li: count: 18 score: 1 wordValue: 0 formattingMultiplier: 0	0.3	-
The changelog is automatically generated by `release-please&…	0 content: p: count: 39 score: 1 code: count: 2 score: 1 wordValue: 0.2 formattingMultiplier: 0	1	-

[ 2.395 WXDAI ]

@rndquu

Contributions Overview

View	Contribution	Count	Reward
Issue	Comment	2	2.395

Conversation Incentives

Comment	Formatting	Relevance	Reward
You mean the `HttpError: Too many subrequests` error is …	0.575 content: p: count: 18 score: 1 code: count: 5 score: 1 wordValue: 0.1 formattingMultiplier: 0.25	0.6	0.345
@gentlementlegen @Keyrxng So the issue is that https://github.…	2.05 content: p: count: 81 score: 1 code: count: 1 score: 1 wordValue: 0.1 formattingMultiplier: 0.25	1	2.05

[ 12.25 WXDAI ]

@Keyrxng

Contributions Overview

View	Contribution	Count	Reward
Issue	Comment	1	12.25

Conversation Incentives

Comment	Formatting	Relevance	Reward
Yeah I think we could create a generator function that tests the…	17.5 content: p: count: 173 score: 1 code: count: 2 score: 1 wordValue: 0.1 formattingMultiplier: 1	0.7	12.25

[ 3.5 WXDAI ]

@0x4007

Contributions Overview

View	Contribution	Count	Reward
Review	Comment	2	3.5

Conversation Incentives

Comment	Formatting	Relevance	Reward
Nice to have, but seems tedious for other people to ensure this …	2.7 content: p: count: 27 score: 1 wordValue: 0.1 formattingMultiplier: 1	1	2.7
```suggestion ``` Really doesn't matter …	0.8 content: pre: count: 1 score: 0 code: count: 1 score: 1 p: count: 7 score: 1 wordValue: 0.1 formattingMultiplier: 1	1	0.8