Skip to content

Conversation

@tsg
Copy link
Member

@tsg tsg commented Feb 22, 2025

This is stacked on top of #4. See commits starting with 4faca9b.

This PR adds a simple scheduler / task queue that saves it's data in Postgres.

At the high level, it works like this:

  • "something" periodically calls a Next.js route to give us a clock tick. This can be a vercel cron that fires every 1 minute, or anything else. For local development, I added pnpm run dev-scheduler that calls the tick route every 10 seconds.
  • when this route is called, we look for jobs that are ready to be run. If they are, we start up to 20 of them in parallel. If there are more than 20, we leave the rest for the next run.
  • after a run, we compute when we want the next run to be (e.g. based on the cron expression) and store that in the DB.

This is a simplistic queue but I think will do the job for now. The only thing that i'm not sure is how we'll call that private tick route when deploying as OSS. Maybe actual cron in the docker-container.

Screenshot 2025-02-22 at 23 14 42

@tsg tsg mentioned this pull request Feb 23, 2025
@SferaDev SferaDev added this pull request to the merge queue Feb 24, 2025
Merged via the queue into main with commit 472b7a7 Feb 24, 2025
4 checks passed
SferaDev added a commit that referenced this pull request Feb 24, 2025
Stacked on top of #4 and #5, this adds running the playbooks via
`generateText` calls.

---------

Co-authored-by: Alexis Rico <[email protected]>
@SferaDev SferaDev mentioned this pull request Mar 19, 2025
@SferaDev SferaDev deleted the add_scheduler branch April 9, 2025 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants