Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

framework for durable task execution #1697

Open
jgraettinger opened this issue Oct 12, 2024 · 0 comments
Open

framework for durable task execution #1697

jgraettinger opened this issue Oct 12, 2024 · 0 comments
Assignees

Comments

@jgraettinger
Copy link
Member

jgraettinger commented Oct 12, 2024

We have a lot of present and future control-plane automations that we want to implement. An impediment is the bespoke nature of current per-task-type async task executors. We think we can substantially improve on this to offer:

  • Much higher execution concurrency without holding transactions.
    • While still providing an at-most-one executor guarantee!
  • A more flexible programming model for tasks, modeled on (optionally) long-lived coroutines that can spawn other tasks, send messages, yield messages to their parent, and maintain inner state.
  • Consistent handling of errors, retries, and task execution leases.
  • More focused, type-safe implementations of task executors.

@psFried and I have had many conversations about what this could look like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant