fix: deactivate function calls when we already have the result to avoir multiple round trips #1130
+20
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue:
When using
output_model
in combination with a tool, the LLM repeatedly requests Function Calls, leading to multiple round trips. Here’s a snippet that reproduces the behavior: https://gist.github.com/nlenepveu/dceb6e0f27cf1d897f3768736070f1aaFix:
While the
tool_call_limit
parameter limits the total number of tool executions, there are instances where the LLM requests for multiple tool calls within a single response, and this parameter cannot enforce a single round trip. Since the LLM’s behavior is unpredictable in terms of how many calls it requests in a single response, the currenttool_call_limit
parameter is insufficient.The proposed fix is to modify the default behavior to automatically set the
tool_choice
to "none" after the first round trip. This ensures the LLM doesn't attempt additional tool calls once it has already received the necessary results from the initial tool execution.