Skip to content

[core] Handle 429 and 500 errors from worlds in runtime#966

Open
VaguelySerious wants to merge 5 commits intomainfrom
peter/429-core
Open

[core] Handle 429 and 500 errors from worlds in runtime#966
VaguelySerious wants to merge 5 commits intomainfrom
peter/429-core

Conversation

@VaguelySerious
Copy link
Member

@VaguelySerious VaguelySerious commented Feb 6, 2026

Uses retry-after header for 429 when provided.
500s are limited to 3x retry, doing exponential backoff.

Review with white space off for sanity.

Claude's suggestion for e2e tests, which I think we might do separately. I think we should make some sort of testbench for world errors and how the runtime reacts.

  3. E2E with a fault-injection endpoint (moderate effort, very targeted)

  Add a test-only endpoint to the workbench app (or a middleware) that can be told to return 429/500 for the next N requests. Something like:

  POST /e2e/inject-fault { status: 429, retryAfter: 3, count: 2 }

  Then the e2e test would:
  1. Inject a fault (e.g. "next 2 requests to workflow handler return 429 with Retry-After: 3")
  2. Start a workflow
  3. Assert it completes successfully despite the transient errors
  4. Optionally check logs/traces for the retry events

  This is the most surgical approach but requires adding test-only infrastructure. The world-local queue already makes HTTP requests to the app endpoints, so you could intercept at that
  layer.

Signed-off-by: Peter Wielander <mittgfu@gmail.com>
@changeset-bot
Copy link

changeset-bot bot commented Feb 6, 2026

🦋 Changeset detected

Latest commit: 9f9c30a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 19 packages
Name Type
@workflow/errors Patch
@workflow/world Patch
@workflow/world-vercel Patch
@workflow/core Patch
@workflow/builders Patch
@workflow/cli Patch
workflow Patch
@workflow/world-local Patch
@workflow/world-postgres Patch
@workflow/web-shared Patch
@workflow/world-testing Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Contributor

vercel bot commented Feb 6, 2026

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

🧪 E2E Test Results

Some tests failed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 479 0 38 517
✅ 💻 Local Development 408 0 62 470
✅ 📦 Local Production 408 0 62 470
✅ 🐘 Local Postgres 408 0 62 470
✅ 🪟 Windows 44 0 3 47
❌ 🌍 Community Worlds 31 157 12 200
✅ 📋 Other 120 0 21 141
Total 1898 157 260 2315

❌ Failed Tests

🌍 Community Worlds (157 failed)

mongodb (39 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • instanceMethodStepWorkflow - instance methods with "use step" directive
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (39 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • instanceMethodStepWorkflow - instance methods with "use step" directive
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • instanceMethodStepWorkflow - instance methods with "use step" directive
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (39 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • instanceMethodStepWorkflow - instance methods with "use step" directive
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 43 0 4
✅ example 43 0 4
✅ express 43 0 4
✅ fastify 43 0 4
✅ hono 43 0 4
✅ nextjs-turbopack 46 0 1
✅ nextjs-webpack 46 0 1
✅ nitro 43 0 4
✅ nuxt 43 0 4
✅ sveltekit 43 0 4
✅ vite 43 0 4
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 40 0 7
✅ express-stable 40 0 7
✅ fastify-stable 40 0 7
✅ hono-stable 40 0 7
✅ nextjs-turbopack-stable 44 0 3
✅ nextjs-webpack-stable 44 0 3
✅ nitro-stable 40 0 7
✅ nuxt-stable 40 0 7
✅ sveltekit-stable 40 0 7
✅ vite-stable 40 0 7
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 40 0 7
✅ express-stable 40 0 7
✅ fastify-stable 40 0 7
✅ hono-stable 40 0 7
✅ nextjs-turbopack-stable 44 0 3
✅ nextjs-webpack-stable 44 0 3
✅ nitro-stable 40 0 7
✅ nuxt-stable 40 0 7
✅ sveltekit-stable 40 0 7
✅ vite-stable 40 0 7
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 40 0 7
✅ express-stable 40 0 7
✅ fastify-stable 40 0 7
✅ hono-stable 40 0 7
✅ nextjs-turbopack-stable 44 0 3
✅ nextjs-webpack-stable 44 0 3
✅ nitro-stable 40 0 7
✅ nuxt-stable 40 0 7
✅ sveltekit-stable 40 0 7
✅ vite-stable 40 0 7
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 44 0 3
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 3 0 0
❌ mongodb 5 39 3
✅ redis-dev 3 0 0
❌ redis 5 39 3
✅ starter-dev 3 0 0
❌ starter 4 40 3
✅ turso-dev 3 0 0
❌ turso 5 39 3
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 40 0 7
✅ e2e-local-postgres-nest-stable 40 0 7
✅ e2e-local-prod-nest-stable 40 0 7

📋 View full workflow run

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 0.032s (-25.5% 🟢) 1.007s (~) 0.975s 10 1.00x
💻 Local Nitro 0.032s (-26.0% 🟢) 1.007s (~) 0.975s 10 1.00x
💻 Local Next.js (Turbopack) 0.040s (+3.4%) 1.007s (-0.9%) 0.967s 10 1.24x
🐘 Postgres Express 0.282s (+52.6% 🔺) 1.012s (-0.6%) 0.730s 10 8.79x
🐘 Postgres Nitro 0.352s (+62.7% 🔺) 1.016s (~) 0.664s 10 10.96x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 0.726s (-2.5%) 2.207s (+36.9% 🔺) 1.480s 10 1.00x
▲ Vercel Next.js (Turbopack) 0.787s (-4.8%) 2.134s (+20.7% 🔺) 1.347s 10 1.08x
▲ Vercel Nitro 0.796s (+21.4% 🔺) 2.167s (+34.8% 🔺) 1.371s 10 1.10x

🔍 Observability: Express | Next.js (Turbopack) | Nitro

workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 1.102s (~) 2.005s (~) 0.903s 10 1.00x
💻 Local Nitro 1.104s (-1.1%) 2.005s (~) 0.901s 10 1.00x
💻 Local Express 1.105s (-1.0%) 2.006s (~) 0.901s 10 1.00x
🐘 Postgres Nitro 2.156s (-2.8%) 2.613s (-13.3% 🟢) 0.457s 10 1.96x
🐘 Postgres Express 2.382s (-0.8%) 3.015s (~) 0.632s 10 2.16x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 2.606s (+3.9%) 3.473s (~) 0.866s 10 1.00x
▲ Vercel Nitro 2.655s (+4.3%) 3.807s (+2.1%) 1.152s 10 1.02x
▲ Vercel Express 2.671s (+6.1% 🔺) 3.716s (-0.6%) 1.045s 10 1.02x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 10.726s (~) 11.021s (~) 0.294s 3 1.00x
💻 Local Nitro 10.815s (~) 11.021s (~) 0.206s 3 1.01x
💻 Local Express 10.821s (~) 11.022s (~) 0.201s 3 1.01x
🐘 Postgres Nitro 15.281s (-1.3%) 16.045s (~) 0.764s 2 1.42x
🐘 Postgres Express 20.242s (-0.7%) 21.059s (~) 0.817s 2 1.89x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 19.517s (-2.4%) 20.803s (~) 1.286s 2 1.00x
▲ Vercel Nitro 20.043s (~) 21.172s (+1.5%) 1.129s 2 1.03x
▲ Vercel Next.js (Turbopack) 20.108s (-1.7%) 21.779s (+2.2%) 1.670s 2 1.03x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 25 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 27.207s (~) 28.040s (~) 0.832s 3 1.00x
💻 Local Express 27.436s (~) 28.045s (~) 0.609s 3 1.01x
💻 Local Nitro 27.456s (~) 28.043s (~) 0.586s 3 1.01x
🐘 Postgres Nitro 37.861s (-1.6%) 38.082s (-2.5%) 0.220s 2 1.39x
🐘 Postgres Express 50.282s (~) 50.623s (-0.9%) 0.341s 2 1.85x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 49.304s (-2.1%) 50.647s (-0.9%) 1.342s 2 1.00x
▲ Vercel Nitro 49.934s (~) 51.072s (+0.5%) 1.138s 2 1.01x
▲ Vercel Next.js (Turbopack) 52.063s (+3.1%) 53.424s (+4.0%) 1.361s 2 1.06x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 50 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 56.660s (~) 57.100s (~) 0.440s 2 1.00x
💻 Local Express 57.041s (~) 57.098s (-1.6%) 0.057s 2 1.01x
💻 Local Nitro 57.152s (~) 58.097s (~) 0.945s 2 1.01x
🐘 Postgres Nitro 75.280s (-1.4%) 76.169s (-1.2%) 0.889s 2 1.33x
🐘 Postgres Express 75.374s (-25.1% 🟢) 76.175s (-24.7% 🟢) 0.800s 2 1.33x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 107.496s (~) 108.602s (~) 1.106s 1 1.00x
▲ Vercel Next.js (Turbopack) 108.062s (+1.8%) 109.060s (+2.3%) 0.998s 1 1.01x
▲ Vercel Express 108.842s (+2.5%) 110.762s (-1.5%) 1.920s 1 1.01x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 1.403s (~) 2.005s (~) 0.602s 15 1.00x
💻 Local Next.js (Turbopack) 1.404s (+1.3%) 2.005s (~) 0.601s 15 1.00x
💻 Local Nitro 1.436s (+2.4%) 2.005s (~) 0.569s 15 1.02x
🐘 Postgres Express 1.637s (-27.8% 🟢) 2.078s (-31.0% 🟢) 0.441s 15 1.17x
🐘 Postgres Nitro 1.818s (-19.4% 🟢) 2.145s (-22.0% 🟢) 0.327s 15 1.30x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.850s (+9.1% 🔺) 3.734s (+1.8%) 0.884s 9 1.00x
▲ Vercel Next.js (Turbopack) 2.884s (+4.9%) 3.848s (+2.6%) 0.965s 8 1.01x
▲ Vercel Express 3.070s (+7.1% 🔺) 4.061s (+6.9% 🔺) 0.991s 8 1.08x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 2.541s (+2.9%) 3.006s (~) 0.465s 10 1.00x
💻 Local Express 2.571s (-1.1%) 3.007s (~) 0.436s 10 1.01x
💻 Local Nitro 2.575s (-0.7%) 3.007s (~) 0.432s 10 1.01x
🐘 Postgres Express 8.444s (-7.2% 🟢) 9.274s (-3.1%) 0.830s 4 3.32x
🐘 Postgres Nitro 8.868s (-19.8% 🟢) 9.289s (-20.5% 🟢) 0.421s 4 3.49x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.316s (+4.7%) 4.560s (+14.7% 🔺) 1.244s 7 1.00x
▲ Vercel Express 3.383s (-12.3% 🟢) 4.461s (-4.2%) 1.077s 7 1.02x
▲ Vercel Next.js (Turbopack) 3.755s (+6.5% 🔺) 5.319s (+23.4% 🔺) 1.564s 6 1.13x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.all with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 7.166s (-1.6%) 8.017s (-3.2%) 0.851s 4 1.00x
💻 Local Nitro 7.395s (+2.9%) 8.020s (-1.6%) 0.625s 4 1.03x
💻 Local Next.js (Turbopack) 7.977s (+11.8% 🔺) 8.515s (+9.4% 🔺) 0.538s 4 1.11x
🐘 Postgres Express 51.652s (+5.7% 🔺) 52.106s (+5.8% 🔺) 0.454s 1 7.21x
🐘 Postgres Nitro 52.284s (~) 53.123s (~) 0.839s 1 7.30x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.800s (-74.2% 🟢) 5.240s (-66.8% 🟢) 1.439s 6 1.00x
▲ Vercel Next.js (Turbopack) 3.832s (-64.6% 🟢) 5.261s (-54.5% 🟢) 1.429s 6 1.01x
▲ Vercel Express 4.282s (-62.2% 🟢) 5.870s (-52.5% 🟢) 1.588s 6 1.13x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 1.409s (-1.7%) 2.004s (~) 0.595s 15 1.00x
💻 Local Nitro 1.428s (+0.8%) 2.005s (~) 0.577s 15 1.01x
💻 Local Next.js (Turbopack) 1.461s (+4.8%) 2.005s (~) 0.543s 15 1.04x
🐘 Postgres Express 1.887s (-8.1% 🟢) 2.078s (-10.4% 🟢) 0.192s 15 1.34x
🐘 Postgres Nitro 1.953s (-12.5% 🟢) 2.154s (-16.9% 🟢) 0.200s 14 1.39x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.669s (+5.3% 🔺) 3.604s (+1.3%) 0.935s 9 1.00x
▲ Vercel Express 2.705s (-13.6% 🟢) 3.485s (-17.6% 🟢) 0.780s 9 1.01x
▲ Vercel Next.js (Turbopack) 2.909s (+14.0% 🔺) 3.931s (+8.1% 🔺) 1.022s 8 1.09x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 2.638s (-2.5%) 3.006s (-0.7%) 0.369s 10 1.00x
💻 Local Nitro 2.658s (-1.0%) 3.006s (~) 0.348s 10 1.01x
💻 Local Next.js (Turbopack) 2.796s (+6.2% 🔺) 3.006s (-5.4% 🟢) 0.210s 10 1.06x
🐘 Postgres Nitro 8.707s (-26.2% 🟢) 9.277s (-22.8% 🟢) 0.570s 4 3.30x
🐘 Postgres Express 10.635s (-3.5%) 11.030s (-2.9%) 0.395s 3 4.03x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.108s (-7.5% 🟢) 4.224s (-3.1%) 1.116s 8 1.00x
▲ Vercel Next.js (Turbopack) 3.224s (+1.7%) 4.316s (+8.8% 🔺) 1.091s 7 1.04x
▲ Vercel Express 3.582s (+28.9% 🔺) 4.535s (+21.9% 🔺) 0.953s 7 1.15x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Promise.race with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 7.701s (+2.2%) 8.269s (-3.5%) 0.568s 4 1.00x
💻 Local Express 7.771s (-4.9%) 8.016s (-11.8% 🟢) 0.245s 4 1.01x
💻 Local Nitro 7.774s (-1.8%) 8.017s (-10.1% 🟢) 0.243s 4 1.01x
🐘 Postgres Nitro 50.814s (-2.4%) 51.151s (-3.7%) 0.337s 1 6.60x
🐘 Postgres Express 54.249s (+6.9% 🔺) 55.114s (+7.8% 🔺) 0.865s 1 7.04x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - -

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.417s (-44.8% 🟢) 4.864s (-28.7% 🟢) 1.447s 7 1.00x
▲ Vercel Next.js (Turbopack) 3.430s (-58.4% 🟢) 4.736s (-47.2% 🟢) 1.307s 7 1.00x
▲ Vercel Express 3.553s (-54.6% 🟢) 5.112s (-40.2% 🟢) 1.560s 7 1.04x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 0.147s (+3.5%) 1.001s (~) 0.010s (-29.3% 🟢) 1.015s (-1.0%) 0.868s 10 1.00x
💻 Local Nitro 0.172s (-4.3%) 1.002s (+1.0%) 0.010s (-23.1% 🟢) 1.015s (~) 0.843s 10 1.17x
💻 Local Express 0.173s (-4.3%) 1.002s (+1.0%) 0.010s (-27.7% 🟢) 1.015s (-0.5%) 0.842s 10 1.17x
🐘 Postgres Express 1.229s (-47.6% 🟢) 1.885s (-30.0% 🟢) 0.001s (+550.0% 🔺) 2.014s (-33.2% 🟢) 0.785s 10 8.34x
🐘 Postgres Nitro 1.404s (+11.2% 🔺) 1.740s (-2.2%) 0.001s (+Infinity% 🔺) 2.012s (~) 0.608s 10 9.53x
🐘 Postgres Next.js (Turbopack) ⚠️ missing - - - - -

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.526s (-59.5% 🟢) 3.039s (-48.6% 🟢) 0.553s (+248.7% 🔺) 4.253s (-42.1% 🟢) 1.728s 10 1.00x
▲ Vercel Nitro 2.696s (-53.2% 🟢) 3.074s (-49.7% 🟢) 0.183s (-23.4% 🟢) 3.912s (-45.2% 🟢) 1.216s 10 1.07x
▲ Vercel Next.js (Turbopack) 2.805s (-47.3% 🟢) 3.132s (-28.5% 🟢) 0.185s (~) 4.126s (-34.2% 🟢) 1.321s 10 1.11x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Next.js (Turbopack) 7/12
🐘 Postgres Express 6/12
▲ Vercel Nitro 7/12
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 💻 Local 10/12
Next.js (Turbopack) 💻 Local 10/12
Nitro 💻 Local 10/12
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Starter: Community world (local development)
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)

📋 View full workflow run

- Propagate Retry-After header as WorkflowAPIError.retryAfter on 429 responses
- Add withThrottleRetry wrapper for both workflow and step handlers
- Re-enqueue workflows on 5xx errors with exponential backoff (5s, 30s, 120min)
- Track serverErrorRetryCount in queue payload for retry budgeting
- Expose delaySeconds on QueueOptions interface

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
);
// Short wait: sleep in-process, then retry once
await new Promise((resolve) =>
setTimeout(resolve, retryAfterSeconds * 1000)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we account for function execution time limits specifically in the case of Vercel world? If the serverless fuction is already close to the end of it's limit and the workflow server throws a 429, adding a 10 sec sleep could potentially exceed the function execution limit and the function could get SIGKILLd midway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow layer should never take much more than a few seconds, so I think it's highly unlikely that we'd run into timeouts, so I'm not too worried about this, but technically a concern

@karthikscale3
Copy link
Collaborator

Rest of the code looks good except the one concern I have around function execution limits in the case of vercel world.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants