Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: deactivate function calls when we already have the result to avoir multiple round trips #1130

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nlenepveu
Copy link
Contributor

Issue:

When using output_model in combination with a tool, the LLM repeatedly requests Function Calls, leading to multiple round trips. Here’s a snippet that reproduces the behavior: https://gist.github.com/nlenepveu/dceb6e0f27cf1d897f3768736070f1aa

Fix:

While the tool_call_limit parameter limits the total number of tool executions, there are instances where the LLM requests for multiple tool calls within a single response, and this parameter cannot enforce a single round trip. Since the LLM’s behavior is unpredictable in terms of how many calls it requests in a single response, the current tool_call_limit parameter is insufficient.

The proposed fix is to modify the default behavior to automatically set the tool_choice to "none" after the first round trip. This ensures the LLM doesn't attempt additional tool calls once it has already received the necessary results from the initial tool execution.

@nlenepveu
Copy link
Contributor Author

I'm starting to doubt whether this is the right way to change the behavior identified in my example. We must still allow the LLM to make function calls after the first round trip, because then it might need a different tool.. Let me know your thoughts on this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant