Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resize and update formatting for the data dictionary page tables under Hubble #1082

Closed
wants to merge 9 commits into from
Closed

Resize and update formatting for the data dictionary page tables under Hubble #1082

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4fc846d
Resize data dictionary page tables and update formatting to make them…
harsha-stellar-data Nov 8, 2024
c67ad0b
Added missing columns when compared to BigQuery datasets into the Docs
harsha-stellar-data Nov 8, 2024
332a197
Update getEvents retention window (#975)
janewang Nov 12, 2024
17404c7
Pull stellar-cli docs from latest released version. (#1068)
fnando Nov 12, 2024
ea1ef86
Update getTransactions docs (#1083)
mollykarcher Nov 12, 2024
cb5d6a7
Stellar developer docs for Galexie (#1072)
urvisavla Nov 12, 2024
4acb7e2
Galexie: Fix typo (#1086)
urvisavla Nov 13, 2024
b7e0787
Add link to Grafana Dashboard for Galexie. (#1085)
urvisavla Nov 13, 2024
6069d48
Refine css file to isolate table formatting options to Data Dictionar…
harsha-stellar-data Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 40 additions & 26 deletions docs/data/hubble/data-catalog/data-dictionary/accounts.mdx
View file
Open in desktop

Large diffs are not rendered by default.

60 changes: 37 additions & 23 deletions docs/data/hubble/data-catalog/data-dictionary/claimable-balances.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,40 @@ title: Claimable Balances
sidebar_position: 20
---

| Name | Description | Data Type | Domain Values | Primary Key? | Natural Key? | Partition or Cluster Field? | Required? | Notes |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| balance_id | A unique identifier for this claimable balance. The Balance id is a compilation of \`Balance Type\` + \`SHA-256 hash history_operation_id\` | string | | | Yes | | Yes | The Balance Type is fixed at V0, \`00000000\`. If there is a protocol change that materially impacts the mechanics of claimable balances, the balance type would update to V1. |
| claimants | The list of entries which are eligible to claim the balance and preconditions that must be fulfilled to claim | array[record] | | | | | Yes | Multiple accounts can be specified in the claimants record, including the account of the balance creator. |
| claimants.destination | The account id who can claim the balance | string | | | | | Yes | |
| claimants.predicate | The condition which must be satisfied so the destination can claim the balance. The predicate can include logical rules using AND, OR and NOT logic. | array[record] | | | | | Yes | |
| claimants.predicate.unconditional | If true it means this clause of the condition is always satisfied | boolean | | | | | No | When the predicate is only unconditional = true, it means that the balance can be claimed under any conditions |
| claimants.predicate.abs_before | Deadline for when the balance must be claimed. If a balance is claimed before the date then the clause of the condition is satisfied. | string | | | | | No | |
| claimants.predicate.rel_before | A relative deadline for when the claimable balance can be claimed. The value represents the number of seconds since the close time of the ledger which created the claimable balance | integer | | | | | No | This condition is useful when creating a timebounds based on creation conditions. If the creator wanted a balance only claimable one week after creation, this condition would satisfy that rule. |
| claimants.predicate.abs_before_epoch | A UNIX epoch value in seconds representing the same deadline date as abs_before. | integer | | | | | No | |
| asset_type | The identifier for type of asset code, can be a alphanumeric with 4 characters, 12 characters or the native asset to the network, XLM. | string | credit_alphanum4 credit_alphanum12 native | | | | Yes | |
| asset_code | The 4 or 12 character code representation of the asset on the network | string | | | | | No | |
| asset_issuer | The address of the account that created the asset | string | | | | | No | |
| asset_amount | The amount of the asset that can be claimed | float | | | | | Yes | |
| sponsor | The account address of the sponsor who is paying the reserves for this claimable balance | string | | | | | No | Sponsors of claimable balances are the creators of the balance. |
| flags | Denotes the enabling and disabling of certain balance issuer privileges | integer | 0 - None, Default 1 - Auth Clawback Enabled | | | | Yes | Flags are set by the claimable balance accounts for an asset. When user accounts claim a balance, the flags applied to the asset originate from this account |
| last_modified_ledger | The ledger sequence number when the ledger entry (this unique signer for the account) was modified. Deletions do not count as a modification and will report the prior modification sequence number | integer | | | Yes | cluster | Yes | |
| ledger_entry_change | Code that describes the ledger entry change type that was applied to the ledger entry. | integer | 0 - Ledger Entry Created 1 - Ledger Entry Updated 2 - Ledger Entry Deleted 3 - Ledger Entry State (value of the entry) | | Yes | | Yes | Valid entry change types are 0, and 2 for ledger entries of type \`claimable_balances\`. Once created, a balance cannot be updated. |
| deleted | Indicates whether the ledger entry (balance id) has been deleted or not. Once an entry is deleted, it cannot be recovered. | boolean | | | | | Yes | |
| batch_id | String representation of the run id for a given DAG in Airflow. Takes the form of "scheduled\_\_\<batch_end_date>-\<dag_alias>". Batch ids are unique to the batch and help with monitoring and rerun capabilities | string | | | | | Yes | |
| batch_run_date | The start date for the batch interval. When taken with the date in the batch_id, the date represents the interval of ledgers processed. The batch run date can be seen as a proxy of closed_at for a ledger. | datetime | | | | MONTH partition | Yes | The table is partitioned on batch_run_date. It is recommended to always include the batch_run_date in the filter if possible to help reduce query cost. |
| batch_insert_ts | The timestamp in UTC when a batch of records was inserted into the database. This field can help identify if a batch executed in real time or as part of a backfill | timestamp | | | | | Yes | |
| asset_id | Unique identifier for asset_code, asset_issuer | integer | | | | cluster | No | |
# Claimable Balances

| | |
| --- | --- |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an FYI, leaving the headers blank creates weird formatting on the Data Dictionaries landing page:
image

vs

image

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure @sydneynotthecity @chowbao . We have to create a new scss file and/or write custom code in the mdx files to achieve this. Also, if some other pages add tables, they may have to write custom files again instead of a global setting. So tried this approach but I agree that it does make some other tables look weird. Will make adjustments and see how the approach pans out

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Given the options, if it's too difficult to create custom code to reformat our tables, I prefer staying with the current global settings. It is better to get our tables to conform to the standard layout than to change the standard layout for all docs.

If the width of the tables decreases, you can also opt to drop 1-2 columns. At minimum we need column name, description and type. The rest are nice to have.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the header info from tables to be shows on the Cards. Now only table names will be shown.

Screenshot 2024-11-14 at 3 13 37 PM

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Made changes to address all the review comments.
  • Also, made changes to the css file to apply for,matting changes to the Dat aDictionary pages only with a custom div. Now other pages won't be impacted.

@sydneynotthecity @chowbao Please review and let me know if any further changes are needed

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preview is available here
http://developers-pr1082.previews.kube001.services.stellar-ops.com/

| Primary Key(s) | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are your thoughts on dropping primary key off of the state table dictionaries? For these tables, natural keys are used; there isn't a singular primary key

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah. I thought of keeping the table structure same and kept it with a BLANK.
Will there be a need to create a unique key (like a generated id) for these tables instead of composite keys? If not, we can drop them completely I guess.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's drop them completely. We have no plans to create a composite key right now

| Natural Key(s) | balance_id, asset_type, asset_code |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:the natural keys are balance_id, ledger_entry_change and closed_at

| Partition Field(s) | batch_run_date (MONTH partition) |
| Clustered Field(s) | last_modified_ledger, asset_id |
| Documentation | [dbt docs](https://hubble-261722.appspot.com/#!/source/source.stellar_dbt_public.crypto_stellar.claimable_balances) |

# Column Details

| Name | Description | Data Type | Domain Values | Required? | Notes |
| --- | --- | --- | --- | --- | --- |
| balance_id | A unique identifier for this claimable balance. The Balance id is a compilation of `Balance Type` + `SHA-256 hash history_operation_id` | string | | Yes | The Balance Type is fixed at V0, `00000000`. If there is a protocol change that materially impacts the mechanics of claimable balances, the balance type would update to V1. |
| claimants | The list of entries which are eligible to claim the balance and preconditions that must be fulfilled to claim | array[record] | | Yes | Multiple accounts can be specified in the claimants record, including the account of the balance creator. |
| claimants.destination | The account id who can claim the balance | string | | Yes | |
| claimants.predicate | The condition which must be satisfied so the destination can claim the balance. The predicate can include logical rules using AND, OR and NOT logic. | array[record] | | Yes | |
| claimants.predicate.unconditional | If true it means this clause of the condition is always satisfied | boolean | | No | When the predicate is only unconditional = true, it means that the balance can be claimed under any conditions |
| claimants.predicate.abs_before | Deadline for when the balance must be claimed. If a balance is claimed before the date then the clause of the condition is satisfied. | string | | No | |
| claimants.predicate.rel_before | A relative deadline for when the claimable balance can be claimed. The value represents the number of seconds since the close time of the ledger which created the claimable balance | integer | | No | This condition is useful when creating a timebounds based on creation conditions. If the creator wanted a balance only claimable one week after creation, this condition would satisfy that rule. |
| claimants.predicate.abs_before_epoch | A UNIX epoch value in seconds representing the same deadline date as abs_before. | integer | | No | |
| asset_type | The identifier for type of asset code, can be an alphanumeric with 4 characters, 12 characters or the native asset to the network, XLM. | string | <ul><li>credit_alphanum4</li><li>credit_alphanum12</li><li>native</li></ul> | Yes | |
| asset_code | The 4 or 12 character code representation of the asset on the network | string | | No | |
| asset_issuer | The address of the account that created the asset | string | | No | |
| asset_amount | The amount of the asset that can be claimed | float | | Yes | |
| sponsor | The account address of the sponsor who is paying the reserves for this claimable balance | string | | No | Sponsors of claimable balances are the creators of the balance. |
| flags | Denotes the enabling and disabling of certain balance issuer privileges | integer | <ul><li>0 - None, Default</li><li>1 - Auth Clawback Enabled</li></ul> | Yes | Flags are set by the claimable balance accounts for an asset. When user accounts claim a balance, the flags applied to the asset originate from this account |
| last_modified_ledger | The ledger sequence number when the ledger entry (this unique signer for the account) was modified. Deletions do not count as a modification and will report the prior modification sequence number | integer | | Yes | |
| ledger_entry_change | Code that describes the ledger entry change type that was applied to the ledger entry. | integer | <ul><li>0 - Ledger Entry Created</li><li>1 - Ledger Entry Updated</li><li>2 - Ledger Entry Deleted</li><li>3 - Ledger Entry State (value of the entry)</li></ul> | Yes | Valid entry change types are 0, and 2 for ledger entries of type `claimable_balances`. Once created, a balance cannot be updated. |
| deleted | Indicates whether the ledger entry (balance id) has been deleted or not. Once an entry is deleted, it cannot be recovered. | boolean | | Yes | |
| closed_at | Timestamp in UTC when this ledger closed and committed to the network. Ledgers are expected to close ~every 5 seconds | timestamp | | Yes | |
| ledger_sequence | The sequence number of this ledger. It represents the order of the ledger within the Stellar blockchain. Each ledger has a unique sequence number that increments with every new ledger, ensuring that ledgers are processed in the correct order. | integer | | Yes | |
| batch_id | String representation of the run id for a given DAG in Airflow. Takes the form of "scheduled\_\_\<batch_end_date>-\<dag_alias>". Batch ids are unique to the batch and help with monitoring and rerun capabilities | string | | Yes | |
| batch_run_date | The start date for the batch interval. When taken with the date in the batch_id, the date represents the interval of ledgers processed. The batch run date can be seen as a proxy of closed_at for a ledger. | datetime | | Yes | The table is partitioned on batch_run_date. It is recommended to always include the batch_run_date in the filter if possible to help reduce query cost. |
| batch_insert_ts | The timestamp in UTC when a batch of records was inserted into the database. This field can help identify if a batch executed in real time or as part of a backfill | timestamp | | Yes | |
| asset_id | Unique identifier for asset_code, asset_issuer | integer | | No | |
58 changes: 35 additions & 23 deletions docs/data/hubble/data-catalog/data-dictionary/contract-code.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,38 @@ title: Contract Code
sidebar_position: 30
---

| Name | Description | Data Type | Domain Values | Primary Key? | Natural Key? | Partition or Cluster Field? | Required? | Notes |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| contract_code_hash | Soroban contract code hash | string | | | Yes | Cluster | Yes | |
| contract_code_ext_v | Contract code extention version | integer | | | | | | |
| last_modified_ledger | The ledger sequence number when the ledger entry (this unique signer for the account) was modified. Deletions do not count as a modification and will report the prior modification sequence number | integer | | | | Cluster | Yes | |
| ledger_entry_change | Code that describes the ledger entry change type that was applied to the ledger entry. | integer | | | | | Yes | |
| deleted | Indicates whether the ledger entry (balance id) has been deleted or not. Once an entry is deleted, it cannot be recovered. | boolean | | | | | Yes | |
| batch_id | String representation of the run id for a given DAG in Airflow. Takes the form of "scheduled\_\_\<batch_end_date>-\<dag_alias>". Batch ids are unique to the batch and help with monitoring and rerun capabilities | string | | | | | Yes | |
| batch_run_date | The start date for the batch interval. When taken with the date in the batch_id, the date represents the interval of ledgers processed. The batch run date can be seen as a proxy of closed_at for a ledger. | datetime | | | | | Yes | |
| batch_insert_ts | The timestamp in UTC when a batch of records was inserted into the database. This field can help identify if a batch executed in real time or as part of a backfill | timestamp | | | | | Yes | |
| closed_at | The UNIX timestamp of the sequence number's age | timestamp | | | | Partition | Yes | |
| ledger_sequence | The unsigned 32-bit ledger number of the sequence number's age | integer | | | | | Yes | |
| ledger_key_hash | Ledger key hash used to identify expiring contract data or contract code ledger entries | string | | | | | Yes | |
| n_instructions | Number of instrcutions in contract code | integer | | | | | | |
| n_functions | Number of functions in contract code | integer | | | | | | |
| n_globals | Number of global variables in contract code | integer | | | | | | |
| n_table_entries | Number of table entries in contract code | integer | | | | | | |
| n_types | Number of types in contract code | integer | | | | | | |
| n_data_segments | Number of data segments in contract code | integer | | | | | | |
| n_elem_segments | Number of element segments in contract code | integer | | | | | | |
| n_imports | Number of imports in contract code | integer | | | | | | |
| n_exports | Number of exports in contract code | integer | | | | | | |
| n_data_segment_bytes | Number of data segment bytes in contract code | integer | | | | | | |
# Contract Code

| | |
| --- | --- |
| Primary Key(s) | |
| Natural Key(s) | contract_code_hash |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also needs closed_at

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closed_at is present

| closed_at | The UNIX timestamp of the sequence number's age | timestamp | | | | Partition | Yes | |

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean as a natural key. The composite key for the table is contract_code_hash + closed_at

| Partition Field(s) | closed_at |
| Clustered Field(s) | contract_code_hash, last_modified_ledger |
| Documentation | [dbt docs](https://hubble-261722.appspot.com/#!/source/source.stellar_dbt_public.crypto_stellar.contract_code) |

# Column Details

| Name | Description | Data Type | Domain Values | Required? | Notes |
| --- | --- | --- | --- | --- | --- |
| contract_code_hash | Soroban contract code hash | string | | Yes | |
| contract_code_ext_v | Contract code extension version | integer | | | |
| last_modified_ledger | The ledger sequence number when the ledger entry (this unique signer for the account) was modified. Deletions do not count as a modification and will report the prior modification sequence number | integer | | Yes | |
| ledger_entry_change | Code that describes the ledger entry change type that was applied to the ledger entry. | integer | | Yes | |
| deleted | Indicates whether the ledger entry (balance id) has been deleted or not. Once an entry is deleted, it cannot be recovered. | boolean | | Yes | |
| batch_id | String representation of the run id for a given DAG in Airflow. Takes the form of "scheduled\_\_\<batch_end_date>-\<dag_alias>". Batch ids are unique to the batch and help with monitoring and rerun capabilities | string | | Yes | |
| batch_run_date | The start date for the batch interval. When taken with the date in the batch_id, the date represents the interval of ledgers processed. The batch run date can be seen as a proxy of closed_at for a ledger. | datetime | | Yes | |
| batch_insert_ts | The timestamp in UTC when a batch of records was inserted into the database. This field can help identify if a batch executed in real time or as part of a backfill | timestamp | | Yes | |
| closed_at | Timestamp in UTC when this ledger closed and committed to the network. Ledgers are expected to close ~every 5 seconds | timestamp | | Yes | |
| ledger_sequence | The sequence number of this ledger. It represents the order of the ledger within the Stellar blockchain. Each ledger has a unique sequence number that increments with every new ledger, ensuring that ledgers are processed in the correct order. | integer | | Yes | |
| ledger_key_hash | Ledger key hash used to identify expiring contract data or contract code ledger entries | string | | Yes | |
| n_instructions | Number of instructions in contract code | integer | | | |
| n_functions | Number of functions in contract code | integer | | | |
| n_globals | Number of global variables in contract code | integer | | | |
| n_table_entries | Number of table entries in contract code | integer | | | |
| n_types | Number of types in contract code | integer | | | |
| n_data_segments | Number of data segments in contract code | integer | | | |
| n_elem_segments | Number of element segments in contract code | integer | | | |
| n_imports | Number of imports in contract code | integer | | | |
| n_exports | Number of exports in contract code | integer | | | |
| n_data_segment_bytes | Number of data segment bytes in contract code | integer | | | |
Loading