Skip to content
/ data-api Public

A collection of APIs for fetching useful custom data from Cosmos SDK blockchains. Built using Cloudflare Workers.

License

Notifications You must be signed in to change notification settings

cheqd/data-api

Cosmos SDK: Custom Data APIs

GitHub release (latest by date) GitHub Release Date GitHub license

GitHub release (latest by date including pre-releases) GitHub commits since latest release (by date) GitHub contributors

GitHub Workflow Status GitHub repo size

ℹ️ Overview

Cosmos SDK offers APIs for built-in modules using gRPC, REST, and Tendermint RPC. This project aims to provide simple REST APIs for data that default Cosmos SDK APIs can't provide.

This collection of custom APIs can be deployed as a Cloudflare Worker or compatible serverless platforms.

🔍 API Endpoints & Features

🧮 Total Supply

Endpoints

data-api.cheqd.io/supply/total (also has an API endpoint alias on /)

Response

Just total supply of tokens, in main token denomination (CHEQ instead of ncheq in our case)

Rationale

Cryptocurrency tracking websites such as CoinMarketCap and CoinGecko require an API endpoint for reporting the total supply of tokens in the main/primary token denomination.

While this figure is available from Cosmos SDK's built-in /cosmos/bank/v1beta1/supply/ncheq REST endpoint, this returns a JSON object in the lowest token denomination, which cannot be parsed by CoinMarketCap / CoinGecko.

🔂 Circulating Supply

Endpoint

data-api.cheqd.io/supply/circulating

Response

Circulating token supply, in main token denomination (CHEQ instead of ncheq in our case)

Rationale

Cryptocurrency tracking websites such as CoinMarketCap and CoinGecko require an API endpoint for reporting the circulating supply of tokens in the main/primary token denomination.

This figure is not available from any Cosmos SDK API, because the criteria for determining circulating vs "non-circulating" accounts is defined by CoinMarketCap.

This API calculates the circulating supply by subtracting the account balances of a defined list of wallet addresses ("circulating supply watchlist") from the total supply.

🥩 Total staked supply

Endpoints

data-api.cheqd.io/supply/staked

Response

Overall tokens staked, in CHEQ.

Rationale

Provides the overall amount staked pulled from the block explorer.

🔐 Vesting Account Balance

Endpoint

data-api.cheqd.io/balances/vesting/<address>

Response

Tokens that are still vesting for continuous/delayed vesting accounts, in CHEQ.

Rationale

There is no Cosmos SDK API that returns balances that are yet to be vested for continuous or delayed vesting accounts.

🔒 Vested Account Balance

Endpoint

data-api.cheqd.io/balances/vested/<address>

Response

Tokens that have already vested for continuous/delayed vesting accounts, in CHEQ.

Rationale

There is no Cosmos SDK API that returns balances that are already vested for continuous or delayed vesting accounts.

💸 Liquid Account Balance

Endpoint

data-api.cheqd.io/balances/liquid/<address>

Response

Tokens in continuous/delayed vesting accounts that can be converted to liquid balances, in CHEQ.

Rationale

Tokens in continuous or delayed vesting accounts that can be converted to liquid balances. This is calculated as the sum of the following figures:

  1. "Delegated free" balance (from the /cosmos/auth/v1beta1/accounts/<address> REST API) or vested balance, whichever is higher
  2. "Available" balance (if applicable)
  3. "Reward" balance (if applicable)

💰 Total Account Balance

Endpoint

data-api.cheqd.io/balances/total/<address>

Response

Total account balance for specified account, in CHEQ.

Rationale

The standard Cosmos SDK REST API for account balances returns JSON with the account balances along with its denomination, usually the lowest denomination. This is hard to parse in applications such as Google Sheets (e.g., to monitor the account balance by fetching a response from a REST API directly in Google Sheets). This API returns a plain number that can be directly plugged into such applications, without having to parse JSON.

📊 Identity Analytics

Endpoints

Mainnet
Testnet

Response

Returns the identity transactions data for the specified network in JSON format. By default, the endpoint returns the data for the last 30 days.

If did or resource is not specified, it will both DID and DLR analytics data, sorted by date in descending order. By default, the response is paginated with a limit of 100 results per page. It's harder to paginate through responses on the combined endpoints, so where possible it is recommended to query DIDs and DLRs separately.

Optional Query Parameters

These optional query paramaters can be used to filter the results that are returned from the API.

  • startDate: Start date for the analytics data. If you provide just this parameter, the current date will be considered the end date.
  • endDate: End date for the analytics data. If you provide just this parameter, the start date will be considered as 30 days prior to this date.
  • operationType: Operation type for the analytics data. It accepts the following values:
    • createDid
    • updateDid
    • deactivateDid
    • createResource
  • ledgerOperationType: Ledger operation type for the analytics data. (Only relevant if the underlying operation types on ledger go through a breaking change in version number and you want to filter by specific operation type. Otherwise, use above.)
    • cheqd.did.v2.MsgCreateDid
    • cheqd.did.v2.MsgUpdateDid
    • cheqd.did.v2.MsgDeactivateDid
    • cheqd.resource.v2.MsgCreateResource
  • denom: Friendly denomination used for paying fees, e.g. CHEQ or USDC. Note that by default the results are returned in main denomination, i.e., the on-ledger value in ncheq is convert to CHEQ.
  • ledgerDenom: Ledger denomination used for paying fees, e.g. ncheq or ibc/498A0751C798A0D9A389AA3691123DADA57DAA4FE165D5C75894505B876BA6E4.
  • feePayer: Fee payer address, e.g., cheqd1...xx.
  • success (boolean; default: true): Whether transaction was successful or not. By default, only successful transactions are returned. Allows for specifically fetching failed transaction details.
  • page (default: 1): Page number for pagination.
  • limit (default: 100): Number of results per page.

Rationale

The purpose of this API is to provide more granular data and help us to understand the usage of identity modules and how they are used by the community.

Example

curl -X GET "https://data-api.cheqd.io/analytics/mainnet?startDate=2024-01-01&endDate=2024-01-31&operationType=createDid&denom=CHEQ&&success=true&limit=1"

Response:

{
  "items": [
    {
      "didId": "did:cheqd:mainnet:f1a32c8b-bd76-450d-b5c2-46b1b92db74d",
      "operationType": "createDid",
      "feePayer": "cheqd15yqrljq6h7u8et30hqrwwaz8f9m62jvztlpc9t",
      "amount": 50,
      "denom": "CHEQ",
      "blockHeight": "11709907",
      "transactionHash": "36EEBB3E4551CC9BB1A70FACB9775AB8DD12F5ABF931A60322558C03F91822B9",
      "createdAt": "2024-01-25T19:49:35.093Z",
      "success": true
    }
  ],
  "totalCount": 7,
  "page": 1,
  "limit": 1,
  "totalPages": 7
}

🧑‍💻🛠 Developer Guide

Architecture

This frontend site was developed to work with Cloudflare Workers, a serverless and highly-scalable platform.

Originally, this project was discussed as potentially being deployed using a serverless platform such as AWS Lambda. However, AWS Lambda has a cold-start problem if the API doesn't receive too much traffic or is only accessed infrequently. This can lead to start times ranging into single/double digit seconds, which would be considered an API timeout by many client applications.

Using Cloudflare Workers, these APIs can be served in a highly-scalable fashion and have much lower cold-start times, i.e., in the range of less than 10 milliseconds.

Setup

The recommended method of interacting with this repository is using Cloudflare Wrangler CLI.

Dependencies can be installed using Yarn or any other package manager.

npm install

While our deployment uses Cloudflare Wrangler, the application itself could be modified to run on other platforms with some refactoring.

Configuration

Wrangler CLI uses wrangler.toml for configuring the application. If you're using this for your own purposes, you will need to replace values for account_id, Cloudflare KV bindings, route, etc. for the application to work correctly along with your own Cloudflare API tokens.

Environment variables

The application expects these environment variables to be set on Cloudflare:

  1. TOKEN_EXPONENT: Denominator for token (default 9 for CHEQ token).
  2. REST_API: REST API for a Cosmos/cheqd node to target for queries.
  3. REST_API_PAGINATION_LIMIT: Number of results to fetch in a single query, for queries that require iterating multiple times. (E.g., many account balance queries require this, to be able to get all delegations etc.)
  4. GRAPHQL_API: GraphQL API for a BigDipper explorer instance for some queries. E.g., the GraphQL API for cheqd's block explorer is https://explorer-gql.cheqd.io/v1/graphql.
  5. CIRCULATING_SUPPLY_GROUPS: Number of sub-groups the circulating supply watchlist is split into (see sample JSON file below). This is to ensure that any lookups from APIs can be spaced out.
  6. MARKET_MONITORING_API: Upstream API for running queries from CoinGecko API (see the market-monitoring repository).
  7. WEBHOOK_URL: Zapier webhook URL to send market monitoring data to. Since this is a secret, it's not set in plaintext in wrangler.toml but passed via GitHub Actions secrets.

Cloudflare KV bindings

Cached data for computationally-expensive queries are stored in Cloudflare KV.

  1. CIRCULATING_SUPPLY_WATCHLIST: This KV is pre-populated with a list of addresses to monitor for circulating supply. Initially, the value portion of this can be set to anything, since it will get replaced when periodic cron triggers run to set the account balance breakdown for this account. In case you have a lot of accounts to monitor, we recommend prefixing the key with a group_N prefix which will stagger the API lookup across multiple cron executions.
// Sample watchlist JSON file structure
[
  {
    "key": "group_1:cheqd1...xxx", // Group 1 prefix
    "value": "26-May-2022" // This can be any value, and will be updated with account balance breakdown periodically
  },
  {
    "key": "group_2:cheqd1...xxx", // Group 2 prefix
    "value": "26-May-2022"
  }
]

Tip: There are online converter tools to transform CSV files to JSON files, which is an easy way of converting spreadsheets to JSON.

Bulk-uploading to Cloudflare KV

Entries can bulk-uploaded to Cloudflare KV using Wrangler CLI:

wrangler kv:bulk put <watchlist-file.json> --binding "CIRCULATING_SUPPLY_WATCHLIST" --preview false

Bulk-deleting from Cloudflare KV

Entries can bulk-deleted from Cloudflare KV using Wrangler CLI by providing a JSON file with list of keys to delete. This JSON file should be in the form ["key1", "key2", ...].

wrangler kv:bulk delete <watchlist-delete.json> --binding "CIRCULATING_SUPPLY_WATCHLIST" --preview false

Local Development

Wrangler CLI can serve a preview where the code and KV pairs are served from Cloudflare. This also automatically executes a build to be able to serve up the app.

wrangler dev

This option will bind itself to the preview_id KV namespace binding (if defined).

Wrangler CLI also allows a degree of local development by running the web framework locally, but this option still relies on Cloudflare backend for aspects such as Cloudflare Workers KV.

wrangler dev --local

If you want completely standalone local development, this can achieved using an emulator framework like Miniflare.

Deploy

Modify the required variables in wrangler.toml for publishing to Cloudflare Workers and execute the following command to execute a build and production deployment.

wrangler publish

Other environments can be targetted (if defined in wrangler.toml) by specifying the --env flag:

wrangler publish --env staging

CI/CD deployments can be achieved using the wrangler Github Action. The staging.yml and release.yml Github Actions in this repo provides an example of this can be achieved in practice.

🐞 Bug reports & 🤔 feature requests

If you notice anything not behaving how you expected, or would like to make a suggestion / request for a new feature, please create a new issue and let us know.

💬 Community

Our Discord server is the primary chat channel for our open-source community, software developers, and node operators.

Please reach out to us there for discussions, help, and feedback on the project.

🙋 Find us elsewhere

Telegram Discord Twitter LinkedIn Medium YouTube