LLMClient and AI SDK integration

[gunta]

Hi! First, amazing work of porting to TypeScript.

I was thinking, since the AI SDK seems like the most advanced and "the most common way" to stream LLMs in TypeScript, in particular in the Frontend or BFF, I think it would be very valuable to have some kind of integration with it.

How would that look like?

[dosco]

It's a great idea. Small issue is that don't want to add any dependencies of external packages to the core library so this would have to be built as a separate library.

It also seems the AI SDK is very simple to do we already support streaming which just needs to be integrated with RSC I don't see it doing much more. Also streaming is only useful for chat style apps not for building workflows where parsing the response requires the whole response.

In short I'll a look I agree with you would be very useful

[gunta]

Small issue is that don't want to add any dependencies of external packages to the core library so this would have to be built as a separate library.

Yes, I don't see a problem with that. In order to be easier to contribute it can be another package in this monorepo.

Also streaming is only useful for chat style apps not for building workflows where parsing the response requires the whole response.
I am not sure about this one in particular.
For example, in partial-stream or partial-json-parser, the workflow response might be already useful in a workflow. This will depend on the application, but I can see that it can have multiple use-cases.

You can check a demo of partial parsing here: https://partial.stream/

[dosco]

Thats true I take that back. I'm working on streaming assertions so types and output format can be checked and fixed early saving tokens and improving latency

[dosco]

more than open to this send me a pr.

[karol-f]

Hi guys, do you think preparing monorepo with llm-client and various solutions and integrations (like AI SDK, Slackbot, crawling websites for data etc.) will be useful?

Should it be part of this project or a completely separate one?

[dosco]

i find managing a single repo is easier it keep things in sync. yes we can start adding an integrations library in a mono-repo format with extended examples and usecases where adding external dependencies are fine.

[gunta]

A single repo is usually easier to maintain

[dosco]

we now have an org ax-llm you can create whatever repos you want and if they are useful to the community we can move them into the org if needed currently the only project in the org is ax the core library. I'm not against a mono-repo either lets build whatever we can figure out where to put it later.