How do I use tools with generateObject?

[LehuyH]

https://sdk.vercel.ai/docs/ai-core/tools says:

The generateObject and streamObject AI functions support tool calls for models with such a capability. The tool call and tool results objects are typed.

But how do I pass tools to generateObject? There doesn't seem to be an option for it

[lgrammel]

generateObject is not intended for tool usage. It is an error in the docs, should say generateText and streamText

[LehuyH]

Ok gotcha - I can send a PR to fix those typos if needed

[matovu-farid]

To save time. I think this should be called out in the docs section for generateObject. With the reason as well

[JClackett]

@lgrammel whats the reason its not intended? I have a few examples where it would be super useful, I now have to pass the result of a generateText call with tools into a streamObject call so I can get structured outputs.

[sirily11]

I hope this behavior can be improved in the future, especially for providers like OpenAI. These providers offer built-in tool support along with object generation which could be useful when switching to vercel's sdk.

[lgrammel]

We have added a experimental_output feature to generateText that allows you to create structured information while calling (openai) tools.

https://sdk.vercel.ai/docs/ai-sdk-core/generating-structured-data#structured-output-with-generatetext

[izharikov]

are there any plans to add support of structured output into streamText to enable streaming, structured output and tools calling?

Seems like OpenAI supports this: doc

[dylanle99]

Does this no longer supported? I can't import experimental_output from generateText as well as Output from ai.

[lgrammel]

@dylanle99 are you on the latest versions of the ai sdk?

[gianpaj]

Could somebody please explain the difference between generateObject() and experimental_output for structure outputs when it comes to OpenAI?

1. Also, which API parameters is the AI SDK finally calling? (OpenAI docs on Structured Outputs)

• generateObject()
  • https://sdk.vercel.ai/cookbook/node/generate-object
• Structured outputs ({ experimental_output } from generateText())
  • https://sdk.vercel.ai/docs/ai-sdk-core/generating-structured-data#structured-outputs-with-generatetext-and-streamtext
• Structured answers ({ toolCalls } from generateText())
  • https://sdk.vercel.ai/docs/foundations/agents#structured-answers

[lgrammel]

you can inspect what is sent by looking at result.response.body

• generateObject calls either json mode or tools depending on config; regular tool calling is not supported
• experimental_output uses json mode; tool calls are supported
• structured answers uses tool calls; other tool calls are also supported but it requires prompt engineering

[nabilfatih]

will experimental_output feature in generateText be stable in v5? or in v5 the generateObject can use tool call as well?

[chrisvariety]

(now that v5 is released...) It's still experimental in v5.

[madagra]

Is there some plan to support tool calling in generateObject?

[viktor89]

whenever i add experimental_output my model loses access to any tools provided. Removing experimental_output it has the tools again

[tobiasbueschel]

Having support for tools in generateObject would be great for the following use case:

We're building an agentic application that makes many tools available to users. We also have follow up prompts that are being generated based on previous messages, which tools were used and also what tools are available.

While we've previously used generateText with experimental_output, using generateObject would feel more suitable as we'd like the LLM to create a structured response with suggested follow up prompts while also knowing about all the available tools in our system (but not calling them).

[QuentinMerabet]

Would love to be able to use tools with generateObject too.

[tomkirk]

me too generateObject and tool calling getting a bit stuck here. what is the recommended aproach for both generate object and tool calling? experimental_output doesnt seem to work for me.