Anthropic processor hangs when an output without text returns

[danthegoodman1]

Description

Bug,

If you can convince claude to call a function only (no additional text for it), it seems to hang pipelines:

2024-12-11 06:46:28.084 | DEBUG    | pipecat.services.anthropic:upgrade_to_anthropic:337 - Upgrading to Anthropic: <pipecat.services.anthropic.AnthropicLLMContext object at 0x1372abe90>
[2024-12-11 06:46:28.084 | DEBUG    | pipecat.services.anthropic:_process_context:141 - Generating chat: you are a helpful assistant | [{"role": "user", "content": [{"type": "text", "text": "."}, {"type": "text", "text": "End the call now."}]}, {"role": "assistant", "content": [{"type": "text", "text": "Goodbye, ending the call now."}, {"type": "text", "text": "Goodbye, ending the call now."}, {"type": "tool_use", "id": "toolu_01778CtXCnyf875nQvDV7115", "name": "end_call", "input": {"final_message": "Goodbye, ending the call now."}}]}, {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_01778CtXCnyf875nQvDV7115", "content": "\"the call will now automatically end, do not say ANYTHING additionally, the user already understands the call is exiting\""}]}]](pipecat.services.anthropic:_process_context:141 - Generating chat: NOT_GIVEN [{"role": "user", "content": [{"type": "text",...)

It seems like it might be sending a blank message to TTS, which triggers bot start speaking, but never stop speaking. You can see from these logs that it "starts speaking" nothing, doesn't generate anything additional, and then only stops speaking over a minute later.

We also see this triggering:

pipecat.services.anthropic:_process_context:249 - AnthropicLLMService#0 exception: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'Requests which include `tool_use` or `tool_result` blocks must define tools.'}

When our 2min user idle timer eventually fires.

I suspect the anthropic processor is sending blank text to TTS, that in turn triggers start speaking, but never stops because there are no audio frames to play, and eventually there is some ~1min timeout on that speaking.

If reporting a bug, please fill out the following:

Environment

• pipecat-ai version: 0.0.48
• python version: 3.11
• OS: macos 14.5