Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
Include
model_id
in Router Error Messages for Enhanced DebuggingRelevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/
directory, Adding at least 1 test is a hard requirement - see detailsNote: An existing test (
test_router_fallbacks_with_model_id
) was identified as covering this change, but no new test was explicitly added during the session.Note: No new test was added, so no screenshot is available.
make test-unit
Note:
make test-unit
was not run during the session.Type
🆕 New Feature
🐛 Bug Fix
Changes
This PR enhances the error messages generated by the LiteLLM Router's fallback mechanism to include the specific
model_id
when a model group fails.Why this change?
Previously, error messages only displayed the
model_group
, making it difficult to pinpoint the exact failing model in scenarios where a model group contains multiple deployments or specific model IDs. By including themodel_id
, debugging becomes significantly easier, allowing developers to quickly identify and address issues with individual model deployments.How it works:
The
model_id
is extracted fromkwargs.get("model_info", {}).get("id")
within the router's exception handling. If available, it is appended to the error message. This change is backward compatible; if nomodel_id
is present, the error message format remains unchanged.Example Error Message Enhancement:
Before:
After:
Slack Thread