Skip to content

Conversation

@AlanPonnachan
Copy link
Contributor

Unified Content Filtering Exception Handling

This PR closes #1035

Standardizing how content filtering events are handled across different model providers.

Previously, triggering a content filter resulted in inconsistent behaviors: generic ModelHTTPError (Azure), UnexpectedModelBehavior (Google), or silent failures depending on the provider. This PR introduces a dedicated exception hierarchy to allow users to catch and handle prompt refusals and response interruptions programmatically and consistently.

Key Changes:

  • New Exceptions: Added ContentFilterError (base), PromptContentFilterError (for input rejections, e.g., Azure 400), and ResponseContentFilterError (for output refusals).
  • OpenAI & Azure: Updated logic to raise PromptContentFilterError for Azure's specific 400 error body and ResponseContentFilterError when finish_reason='content_filter'.
  • Google Gemini: Updated _process_response to raise ResponseContentFilterError instead of UnexpectedModelBehavior when safety thresholds are triggered.
  • Anthropic: Added mapping for refusal stop reasons to raise ResponseContentFilterError.
  • Tests: Added comprehensive tests covering synchronous and streaming scenarios for OpenAI, Google, and Anthropic in tests/models/.

Example Usage:

from pydantic_ai import Agent
from pydantic_ai.exceptions import ContentFilterError

agent = Agent('openai:gpt-4o')

try:
    # If the prompt or generation triggers a safety filter
    await agent.run("Generate unsafe content...")
except ContentFilterError as e:
    # Catches both PromptContentFilterError and ResponseContentFilterError
    print(f"Request halted by safety filter: {e}")

@rahim-figs
Copy link

Is it possible to handle AWS Bedrock as well? Thanks.

Copy link
Collaborator

@dsfaccini dsfaccini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @AlanPonnachan thank you for the PR! I've requested a couple small changes, let me know if you have any questions

@dsfaccini
Copy link
Collaborator

one more thing I missed, please include the name of the new exception in the fallback model docs

ModelAPIError, which includes ModelHTTPError and ...

@AlanPonnachan
Copy link
Contributor Author

@dsfaccini Thank you for the review. I’ve made the requested changes.

@dsfaccini
Copy link
Collaborator

hey @AlanPonnachan thanks a lot for your work! It looks very good now, I requested a couple more changes but once that's done and coverage passes I think the PR will be ready for merge.

@AlanPonnachan
Copy link
Contributor Author

@dsfaccini Thanks again for the review! I’ve applied the requested changes. Test coverage is now at 100%.

Copy link
Collaborator

@DouweM DouweM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlanPonnachan Thanks for working on this! My main concern is that this doesn't actually make it consistent for all models that respond with finish_reason=='content_filter', just for Anthropic/Google/OpenAI.

chat.chat_completion.Choice(
finish_reason='content_filter',
index=0,
message=chat.ChatCompletionMessage(content='', role='assistant'),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? I'd want it to result in a ModelResponse with 0 parts, and this may result in a TextPart(content='')

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be cleaner to just build that ModelResponse directly...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a ModelResponse directly would be cleaner but _completions_create is strictly typed to return ChatCompletion (or AsyncStream).
I updated to constructing a ChatCompletion with content=None (not '') which will successfully results in a ModelResponse with 0 parts via _process_response.

@AlanPonnachan
Copy link
Contributor Author

@DouweM , I’ve made a few changes. Let me know your thoughts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Content Filtering Exception Handling

4 participants