Frequently getting AI_APICallError: Cannot connect to API: other side closed

[karl-spark]

I've been frequently getting this "AI_APICallError: Cannot connect to API: other side closed" when I use generateObject and it seems like a timeout issue. Becuase it always time out at around 270 seconds

This request normally takes only 90 seconds to complete. Input is about 38K tokens. I was just hoping if someone could share more insight about this or someone else also experiences the same thing.

Thanks!

[ahuaiqom]

Hi @karl-spark 👋

The failure at ~270s is almost certainly an upstream idle/processing timeout (“other side closed”) from the provider/proxy in front of grok-4-fast-reasoning. Your call runs long (38k tokens in, heavy reasoning), and the server closes the HTTP/2 stream before it finishes.

**What to try (works in practice):

1. Stream the response (keeps the connection active and survives idle limits)**

ts
import { streamObject } from "ai";
const { partialObjectStream, toFinalStream } = await streamObject({
  model: xai("grok-4-fast-reasoning"),
  schema,
  input,
  maxTokens: 8_000,          // cap output
});
for await (const delta of partialObjectStream) {
  // emit partials to client; keeps the socket alive
}

2. Reduce work per call

• Trim context (embed/retrieve only the top chunks).

• Lower reasoning effort / temperature; cap maxTokens; add stop if possible.

• Split the task into smaller sub-calls (map/reduce style).

3. Harden the client

ts
const res = await generateObject({
  model: xai("grok-4-fast-reasoning"),
  input,
  maxRetries: 2,                  // brief retries for transient closes
  // If your runtime allows, increase fetch timeout a bit:
  // fetch: { timeout: 300_000 }
});

4. Runtime limits
If you’re on a serverless runtime, ensure you stream from the route handler; non-streaming responses can hit platform timeouts even if the model is still generating.

5. If it persists
Capture the provider request ID/timestamp and open a ticket with x.ai—they can confirm the exact upstream timeout.

TL;DR: It’s a long-running call getting closed upstream. Stream, cap tokens, and shrink the prompt (or split the job).

[karl-spark]

Thank you for this answer! Will definitely try these! I tried the exact payload but used generateText and it goes way faster. Then I use a smaller generateObject to parse the output to my desired object form

[github-actions[bot]]

This discussion was automatically locked because it has not been updated in over 30 days. If you still have questions about this topic, please ask us at community.vercel.com/ai-sdk