Some issues getting started with groq

[Madd0g]

Hey, looks very neat, like what I'm seeing so far!

Few issues:

1. Ollama - my ollama server is running on another computer, need to be able to provide the base url.
2. So I switched to Groq - tried to use ReAct with a function call and it worked - however seems weird that it's mentioning a task_done function that is not defined (not passed as a possible tool). Another thing that isn't really clear is the output - is the final answer parsed out with regex?
3. I tried to use the agents example

a. it seems like it's asking a bunch of physics questions, is the expectation that the AI will do parallel function calls for this?

b. I shortened it to one question, but I see the response is always an empty string (I added logging in the openai/api.ts file and I don't even see a separate request that the researcher agent is doing)

Maybe I messed something up? Can you see something wrong with my example?

import {
  AI,
  Agent,
  GroqArgs,
} from "llmclient";

const ai = AI("groq", {
  apiKey: "mykey",
  config: {
    model: "mixtral-8x7b-32768",
    // model: "llama3-70b-8192",
    maxTokens: 2000,
  },
} as GroqArgs);

ai.setOptions({ debug: true });

const researcher = new Agent(ai, {
  name: "researcher",
  description: "Researcher agent",
  signature: `physicsQuestion "physics questions" -> answer "reply in bullet points"`,
});

const summarizer = new Agent(ai, {
  name: "summarizer",
  description: "Summarizer agent",
  signature: `text "text so summarize" -> shortSummary "summarize in 5 to 10 words"`,
});

const agent = new Agent(ai, {
  name: "agent",
  description: "Agent",
  signature: `question -> answer`,
  agents: [researcher, summarizer],
});

async function summarize(question: string) {
  const res = await agent.forward({ question });
  console.log(">", res);
}

summarize(
  `Research first and then summarize: Why is the centrifugal force talked about so much if it's not real?`
);

what I see is:

[
 // { ... },
 {
    role: 'assistant',
    content: undefined,
    name: undefined,
    tool_calls: [
      {
        id: 'call_q8sf',
        type: 'function',
        function: {
          name: 'researcher',
          arguments: `{"physicsQuestion":"Why is the centrifugal force talked about so much if it's not real?"}`
        }
      }
    ]
  },
  { role: 'tool', content: '', tool_call_id: 'call_q8sf' }
],

[dosco]

1. new AI("ollama", { url: 'http://localhost:11434' })

2. Yes we ise a task_done function to allow the llm to signal completion of a multi-step tool use task. This was a work-around for some less capable llms to help them return results. We're actively improving things so this can change in the near future. Yes the result values output fields are parsed out with regex as they follow a fixed format.

3. Agents - It's a demo example where the functions are supposed to fetch data from a database but we used another llm to simulate this instead. You can change it to your needs it's only meant to show you how agents can be created using signatures and how agents can be used by other agents, etc

a. Yes, we support parallel function calling where available such as with openai
b. You can debug with ai.setOptions({ debug: true }) it will provide you the entire trace on console.

[dosco]

Lots of fixes in the latest version give it a try.

[Madd0g]

I'm seeing the same thing with the new version, I see yellow lines for agent tool-calls but not actual API calls for those agents. if I go into openai/api.js and add console.log for reqValue, I see empty content for these tool calls.

if you think the agents example should work, maybe I'll try to start over, it is possible I messed something up.

anyhow, I think a function call that doesn't actually execute for some reason should be logged somehow.

[karol-f]

if I go into openai/api.js and add console.log for reqValue, I see empty content for these tool calls.

Can you use e.g. https://httptoolkit.com to be sure that there are no calls?

[Madd0g]

It says zero config https interception? I'll try to find some time to try something like that. I'm more inclined to dig deeper into the code to find why it's not working lol

FWIW, It didn't look like other code would make API calls - I get the logs printed for the "main agent", if all agents are using the same ai, all requests should go through the same place, right?

I'm using openai gpt4 and after the yellow line with the function calls, the next log line (which is wrongly calling the main agent again) is almost immediate. An error could be that fast, I guess, but not a real response.

[dosco]

i'll take a look today with groq

[dosco]

With ollama I not seeing support for function calling in their api. With groq I tried today the best example to test function calling is food-search.ts it worked fine with even the 'mixtral-8x7b-32768 model even though llama3-70b-8192 is the one they recommend for function calling.

{
  reason: "Based on your query, I have used the 'findRestaurants' tool to look for sushi restaurants in San Francisco with outdoor seating and a price range of $$. The tool returned three options, including a sushi restaurant and two other options.",
  restaurant: 'Sukiyabashi Jiro',
  priceRange: '$$'
}

[dosco]

looking into this

[dosco]

Make several fixes to agents and Groq give it a try. 9.0.11

[Madd0g]

same result with both groq and gpt4o:

this is what I see in the messages array (it's still not actually calling the researcher/summarizer):

    {
      role: 'assistant',
      content: undefined,
      name: undefined,
      tool_calls: [
        {
          id: 'call_4zbk',
          type: 'function',
          function: {
            name: 'physicsResearcher',
            arguments: `{"physicsQuestion":"Why is gravity not a real force? Why is light pure energy? Why is physics scale invariant? Why is the centrifugal force talked about so much if it's not real?"}`
          }
        }
      ]
    },
    { role: 'tool', content: '', tool_call_id: 'call_4zbk' }

[dosco]

Sorry this was on me, growing pains I had missed an obvious bug. Thanks for the persistance. Should be fixed in the latest release 9.0.14

[Madd0g]

Thanks for the persistance

😅

Thanks, confirmed that the example is working for me now.