Cerebras messes with tools

[mwufi]

Using this configuration works

    assistant = VoiceAssistant(
        vad=ctx.proc.userdata["vad"],
        stt=deepgram.STT(),
        llm=openai.LLM(),
        tts=cartesia.TTS(voice="41f3c367-e0a8-4a85-89e0-c27bae9c9b6d"),
        chat_ctx=initial_ctx,
        fnc_ctx=func_ctx,
    )

But if we replace openai.LLM with

llm=openai.LLM(
        base_url="https://api.cerebras.ai/v1",
        api_key=os.environ.get("CEREBRAS_API_KEY"),
        model="llama3.1-8b",
)

then the agent is no longer able to see the results of tool calls

[billyg88]

Shouldn't you be using with_cerebras on your LLM declaration?

[imohitmayank]

Facing similar issue, even with officia Cerebras example and using both 8b and 70b model, function call is never happening. Is this bcoz of model or the some code issue?

[jayeshp19]

Hi @mwufi @imohitmayank,

I've tried reproducing these issues, and it seems Cerebras models aren't optimal for tool use. Here are my observations:

• llama3.1-8b: Doesn't handle tool calls properly.
• llama3.1-70b: Generates same tool call even after receiving a tool response.

Hope this helps clarify!

[davidzhao]

I've talked to the Cerebras team and they've confirmed these will be improved very soon, currently planned in the next few weeks! Stay tuned and we'll update here when it's resolved.

[imohitmayank]

Btw function call is much better for llama-3.3-70b.