Add event for API limit usage monitoring

[sunlei]

Feature Request

We primarily execute strategies on cryptocurrency exchanges, all of which impose rate limits. These limits typically fall into two categories:

• IP-based limits
• Account-based limits

Those informations is typically included in the response headers.
I remember there used to be a show_usage option in the Binance adapter, which was used to output limit usage information to log.

In high-frequency strategies, these rate limits are critical and require constant monitoring. Therefore, I propose adding a new event type to communicate limit usage data, which could be passed via callbacks like on_data.

What do you think about this idea?

[cjdsellers]

Hi @sunlei

This is an interesting one. It would help to understand the potential use cases in more detail

Some questions:

• How would these sort of stats compliment the rate limiters we have in place already?
• For normalizing the type, which APIs would we refer to (are they common enough)?
• Would it be a custom data type to subscribed for?
• Would you intend on adding this for Bybit only?

[sunlei]

@cjdsellers

• How would these sort of stats compliment the rate limiters we have in place already?
  • My current idea is to keep this feature independent of the existing rate limiters. Due to the complexity of aligning the existing rate limiters with the rate limit information returned by exchanges, it might become overly complicated. Having it as a separate mechanism allows strategies to dynamically adjust based on the provided information, which could offer more flexibility.
• For normalizing the type, which APIs would we refer to (are they common enough)?
  • Binance: https://binance-docs.github.io/apidocs/futures/en/#limits
  • OKX: https://www.okx.com/docs-v5/en/#overview-rate-limits
  • Bybit: https://bybit-exchange.github.io/docs/v5/rate-limit
• Would it be a custom data type to subscribed for?
  • Perhaps implementing a subscribe_limit_usage method? Similar to subscribing to market data, but the data here would be derived from the response headers of other requests.
• Would you intend on adding this for Bybit only?
  • This might be needed for most cryptocurrency exchanges.

[cjdsellers]

Thanks, this seems reasonable, and I can see how it could be valuable.

Do you have any thoughts on the struct—specifically the fields and types we'd want to include?

One option could be to start with it as a custom type, and later make it first-class with a subscribe_limit_usage method on Actor.

[sunlei]

@cjdsellers

I haven’t thought through the specifics of the data structure yet. It will require abstracting information from various exchanges to create a unified model.

Currently, I’m working on adding WebSocket-based order functionality for Bybit. Once that is completed, I can start considering how to design the data structure.