Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Analytics crate + downtime metrics between trades for a given time period #1021

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

epappas
Copy link

@epappas epappas commented Nov 22, 2024

Motivation

We wanted to understand the "downtime" between trades as a critical KPI for evaluating the performance and responsiveness.

Currently we're lacking a crate focusing on the analytics part for the future metrics we want to expand on, limiting visibility into creating KPIs.

Solution

This proposed change introduces the downtime metric (min, max & avg) for the time intervals between consecutive trades, irrespective of the orders (gathers all).

Example:

let (avg, min, max) = analytics.calculate_downtime_between_trades(Some((start, end))).await;

println!("Average downtime: {} seconds", avg);
println!("Minimum downtime: {} seconds", min);
println!("Maximum downtime: {} seconds", max);

and at the cli:

cargo run -p rain_orderbook_cli analytics downtime --start=01-11-2024 --end=20-11-2024 --subgraph-url=<url>

The current implementation relies in gathering all orders and then sorting the trades in order to calculate the needed time deltas. A potential enhancement is to go directly on chain and get the transfers, as we rely on subgraph crate for now, which gives us the orders.

Checks

By submitting this for review, I'm confirming I've done the following:

  • made this PR as small as possible
  • unit-tested any new functionality
  • linked any relevant issues or PRs
  • included screenshots (if this involves a front-end change)

use rain_orderbook_subgraph_client::{OrderbookSubgraphClient, OrderbookSubgraphClientError};

#[async_trait]
pub trait OrderbookSubgraphClientTrait {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm creating this for two reasons:

  1. To make the code more testable
  2. To not pollute the OrderbookSubgraphClient with analytics-related methods

}

fn create_mock_trade(timestamp: u64) -> Trade {
Trade {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: IMO this should had been provided as a mock interface from the subgraph crate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense - can you move it over there?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can, but this particular block is very ugly and opinionated for this set of unit tests. Also out of scope of this PR i'd say.

What I's thinking was some sort of a testing crate that creates fake trades & orders for a given scenario.

let orders = self.orders_list_all().await?;

let mut all_trades = Vec::new();
for order in orders {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the performance output of this, but to fix that, would had been out of the scope of the PR.

all_trades.extend(trades);
}

all_trades.sort_by(|a, b| {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorting the trades, as it's not guaranteed that the orders will give me chronologically sorted trades.

@epappas
Copy link
Author

epappas commented Nov 30, 2024

change in the original API. Added --threshold cli param (eg --threshold=1m) and total & count downtime metrics.

example:

rain_orderbook_cli analytics downtime --threshold=35s --start=01-11-2024 --end=20-11-2024 --subgraph-url=***
Average downtime: 1356.39 seconds
Minimum downtime: 36.00 seconds
Maximum downtime: 5206.00 seconds
Number of occurrences: 1207
Total downtime: 1637160.00 seconds

using the analytics crate

let (avg, min, max, count, total) = analytics.calculate_downtime_between_trades(period, threshold_secs).await;

println!("Average downtime: {:.2} seconds", avg);
println!("Minimum downtime: {:.2} seconds", min);
println!("Maximum downtime: {:.2} seconds", max);
println!("Number of occurrences: {}", count);
println!("Total downtime: {:.2} seconds", total);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants