Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ingest/cdp: get latest ledger as default for unbounded range #5495

Open
sreuland opened this issue Oct 22, 2024 · 1 comment
Open

ingest/cdp: get latest ledger as default for unbounded range #5495

sreuland opened this issue Oct 22, 2024 · 1 comment

Comments

@sreuland
Copy link
Contributor

sreuland commented Oct 22, 2024

What problem does your feature solve?

for uses cases of cdp.ApplyLedgerMetadata where caller just wants it to start a range at whatever is latest ledger of network, requires the caller to obtain the latest ledger manually from history archives first and pass that as range.from

What would you like to see?

When caller of cdp.ApplyLedgerMetadata passes UnboundedRange.from=0, cdp.ApplyLedgerMetadata should detect and acquire the latest ledger internally and substitute that in the UnboundedRange.from value passed to BufferedStorageBackend.

An implementation choice needs be determined on how/which 'latest' ledger is obtained:

  1. the checkpoint ledger from target network based on history archives
  2. the greatest ledger sequence that exists on the Datastore

If second option is chosen to use Datastore, then this ticket does become effectively blocked on #5498 which provides the changes on Datastore to facilitate.

What alternatives are there?

callers of cdp.ApplyLedgerMetadata that want to start streaming from current network ledger, must manually call HistoryArchive.GetLatestLedgerSequence() first to obtain the latest sequence.

@sreuland
Copy link
Contributor Author

It would be extra cool if Datastore could be queried to provide its highest and lowest ledger sequences currently available on it, then this type of behavior for clients to initiate unbounded from latest data point would be trivial:
#5498

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Backlog
Development

No branches or pull requests

3 participants