🔄 Upstream Sync: LiteLLM v1.81.0-stable #88
Conversation
…tore [Feature] UI - Spend Logs: Settings Modal
…rror_message [Feature] Add error_message Search in Spend Logs Endpoint
…ssthroughs to stop working for router models (BerriAI#19967) * fix(vertex_ai): replace custom model names with actual Vertex AI model names in passthrough URLs (BerriAI#19948) When the passthrough URL already contains project and location, the code was skipping the deployment lookup and forwarding the URL as-is to Vertex AI. For custom model names like gcp/google/gemini-2.5-flash, Vertex AI returned 404 because it only knows the actual model name (gemini-2.5-flash). The fix makes the deployment lookup always run, so the custom model name gets replaced with the actual Vertex AI model name before forwarding. * add _resolve_vertex_model_from_router * fix: get_llm_provider * Potential fix for code scanning alert no. 4020: Clear-text logging of sensitive information Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: michelligabriele <[email protected]> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…t in router (BerriAI#19969) * feat: List all available search tools configured in the router. * add debugging search API * add debugging search API
…component [Feature] UI - Tables: Reusable Table Sort Component
…m_search [Feature] UI - Logs: Adding Error message search to ui spend logs
…eams (BerriAI#19972) * fix: create_vector_store_in_db * add team/user to LiteLLM_ManagedVectorStore * add _check_vector_store_access * add new fields * test_check_vector_store_access * add vector_store/list endpoints * fix code QA checks
…7`, `z-ai/glm-4.7-flash`, and `minimax/minimax-m2.1`. to model prices and context window (BerriAI#19938) Co-authored-by: Rushil Chugh <Rushil>
…an_29 fix gemini gemini-robotics-er-1.5-preview entry
…hropic (BerriAI#19896) * fix(vertex_ai): convert image URLs to base64 in tool messages for Anthropic Fixes BerriAI#19891 Vertex AI Anthropic models don't support URL sources for images. LiteLLM already converted image URLs to base64 for user messages, but not for tool messages (role='tool'). This caused errors when using ToolOutputImage with image_url in tool outputs. Changes: - Add force_base64 parameter to convert_to_anthropic_tool_result() - Pass force_base64 to create_anthropic_image_param() for tool message images - Calculate force_base64 in anthropic_messages_pt() based on llm_provider - Add unit tests for tool message image handling * chore: remove extra comment from test file header
* fix(proxy_server): pass search_tools to Router during DB-triggered initialization * fix search tools from db * add missing statement to handle from db * fix import issues to pass lint errors
[Infra] Remove _experimental/out routes from gitignore + UI Build
…_usage_report [Feature] UI - Usage Export: Breakdown by Teams and Keys
🔧 Fix: Redis Session Key Mismatch (commit c284cde)IssueAI tests were failing with context loss errors:
Root CauseThe original CARTO Redis session patch (PR #5, commit # STORE (uses raw response_id):
name=f"litellm_patch:session:{response_id}"
# GET (decoded the ID first - WRONG):
actual_request_id = decode_previous_response_id_to_original_previous_response_id(previous_response_id)
name=f"litellm_patch:session:{actual_request_id}"This caused sessions to be stored but never retrieved because the keys didn't match. FixRemoved the ID decoding on the GET side - now both store and retrieve use the same raw ID format: # GET (now uses raw previous_response_id):
name=f"litellm_patch:session:{previous_response_id}"Files Changed
VerificationDocker image rebuild triggered. Re-run cloud-native integration tests after image is updated. |
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
✅ CARTO Feature Analysis Complete
Overall Assessment: PASS 📋 Full details in PR description above. |
The Response API wasn't storing sessions in Redis for streaming requests, only for non-streaming. This caused context to be lost when using previous_response_id with streaming responses. Changes: - Add _store_session_in_redis method to streaming iterator - Store full conversation history immediately when stream completes - Pass litellm_completion_request to streaming iterator for message history - Ensures streaming behaves identically to non-streaming for session storage This fixes the timing issue where a delay was needed between requests to allow batch processing to store sessions. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
🔧 Fix: Streaming Redis Session Storage (commit 4b459a8)IssueThe Redis session storage patch was completely missing from the streaming iterator in the upstream-sync branch. This caused:
Root CauseDuring the upstream sync conflict resolution, the streaming Redis patch from PR #5 was lost. The carto/main branch had 3 Redis references in streaming_iterator.py, but upstream-sync had 0. FixCherry-picked commit
Files Changed
VerificationDocker image will rebuild. Re-run cloud-native integration tests to verify coherence is restored. Note: This is a fundamental CARTO feature that enables conversation context persistence for the AI Agent. |
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
✅ CARTO Feature Analysis Complete
Overall Assessment: PASS 📋 Full details in PR description above. |
Addresses progressive OCI streaming by handling missing required fields in tool calls and content items before Pydantic validation, preventing errors. Specifically, it assigns empty strings or dictionaries to missing 'arguments', 'id', and 'name' in tool calls. It also adds empty 'text' or 'imageUrl' fields when missing in content items of type TEXT or IMAGE, respectively.
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
✅ CARTO Feature Analysis Complete
Overall Assessment: PASS 📋 Full details in PR description above. |
|
Restores the complete OCI Gemini tool calling fix (PR #68) that was lost during the upstream sync to v1.81.0-stable. Changes from carto/main: - transformation.py: Generate UUIDs for missing OCI tool call IDs instead of empty strings (which caused silent tool call drops) - streaming_iterator.py: Add tool call accumulation and proper Responses API event emission (_handle_tool_call_delta, _emit_function_call_done_events) Without this fix, OCI Gemini models return empty actions[] because tool calls with empty IDs are silently dropped during response aggregation in stream_chunk_builder.
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
✅ CARTO Feature Analysis Complete
Overall Assessment: PASS 📋 Full details in PR description above. |
…, #58 Port 5 CARTO features that were silently dropped during the v1.81.0 upstream sync due to class hierarchy refactoring: - Add SnowflakeStreamingHandler for missing 'created' field and Claude-format tool_use -> OpenAI tool_calls transformation - Add _transform_messages() for role="tool" -> Snowflake content_list - Enhance _transform_tool_choice() to convert strings to Snowflake object format (e.g. "auto" -> {"type": "auto"}) - Add get_model_response_iterator() override - Default tool_choice to "auto" when tools present in Responses API Also adds .github/carto-features.yml manifest to track critical customizations and prevent future sync regressions.
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
✅ CARTO Feature Analysis Complete
Overall Assessment: PASS 📋 Full details in PR description above. |
…ortex endpoint The upstream v1.81.0 refactored Snowflake URL construction into a two-step build (_get_api_base + get_complete_url). When api_base already contains the full endpoint path (e.g. from CARTO platform config), the path gets appended twice, causing 404 errors on ded-07. Skip path construction if api_base already contains cortex/inference:complete.
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
✅ CARTO Feature Analysis Complete
Overall Assessment: PASS 📋 Full details in PR description above. |
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
|
| Decision | Count |
|---|---|
| Upstream Substitutes | 1 |
| Customized Upstream | 4 |
| Preserved CARTO | 16 |
| Incorrectly Dropped | 1 |
Overall Assessment: NEEDS_ATTENTION
🔧 Auto-fix enabled: The fix job will run next to restore dropped features.
📋 Full details in PR description above.
🔧 CARTO Feature Fix StartedRestoring 1 incorrectly dropped CARTO feature(s). |
Features restored: - [PR #90] Multi-architecture Docker builds for non_root image Changes: - Created docker-build-multiarch.yaml reusable workflow with matrix builds for linux/amd64 (ubuntu-latest) and linux/arm64 (ubuntu-24.04-arm) - Updated carto-ghcr-deploy.yaml to use the reusable workflow - Fixed paths trigger (was incorrectly pointing to ghcr_carto_deploy.yaml) - Updated carto-release.yaml to use the reusable workflow Integration approach: - Direct restoration of CARTO's multi-arch build capability - No upstream equivalent existed - Uses native GitHub runners (no QEMU emulation) - Each arch builds natively, pushes by digest, then merges into manifest Verified: YAML syntax validation passed Co-Authored-By: Claude Opus 4.5 <[email protected]>
🔧 CARTO Feature Fix CompleteSummaryRestored 1 CARTO feature (PR #90 multi-arch Docker builds) using systematic analysis and direct restoration Decisions MadePR #90: feat(docker): multi-arch builds (AMD64 + ARM64) for non_root imageDecision: CARTO's PR #90 added multi-architecture Docker build capability that was completely removed during upstream sync. The feature is independent infrastructure with no upstream equivalent. Direct restoration was performed by recreating the docker-build-multiarch.yaml workflow and updating both carto-ghcr-deploy.yaml and carto-release.yaml to use it. Next Steps:
|
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
✅ CARTO Feature Analysis Complete
Overall Assessment: PASS 📋 Full details in PR description above. |
20 participants
🔄 Upstream Sync: LiteLLM v1.81.0-stable
Syncs CARTO's LiteLLM fork with upstream stable release v1.81.0-stable.
1.79.1→v1.81.0-stableCaution
Use "Create a merge commit" only. Squashing destroys upstream history and breaks future syncs.
🧪 Pre-Merge Checklist
pyproject.tomlversion matches upstream📊 Release Information (click to expand)
v1.81.0-stable1.79.1🔀 Branch Flow (click to expand)
BerriAI/litellm:mainmerged intoCartoDB/litellm:mainupstream-sync/v1.81.0-stableupstream-sync/v1.81.0-stable→carto/main📝 CARTO-Specific File Guidelines (click to expand)
When reviewing or resolving conflicts:
✅ Keep CARTO Versions (Ours)
.github/workflows/carto_*.yaml- CARTO workflows.github/workflows/carto-*.yml- CARTO workflowsCARTO_*.md,docs/CARTO_*.md- CARTO documentation🔄 Accept Upstream (Theirs)
pyproject.toml- Version fieldlitellm/- Core library codetests/- Upstream testsrequirements.txt- DependenciesDockerfile,docker/Dockerfile.non_root- CARTO customizationsMakefile- Check# CARTO:sections🔧 Conflict Resolution (click to expand)
If this PR has conflicts:
Option 1: Automated (Recommended)
The carto-upstream-sync-resolver workflow triggers automatically.
What it does:
carto/main→ ✏️ Resolves conflicts → 🧪 Runs tests → 📌 Pushes to this PRYou just need to: Wait for resolution commits, verify CARTO customizations, merge.
Option 2: Manual Resolution
📚 Documentation Links (click to expand)
🤖 This PR was automatically created by the carto-upstream-sync workflow.
Warning
The automated resolver and analyzer incorrectly classified PR #68 as fully preserved. CARTO's content item padding code was silently dropped, causing production OCI streaming failures.
What was dropped
PR #68 added two defensive padding blocks to
_handle_generic_stream_chunk()inlitellm/llms/oci/chat/transformation.py:arguments,id,namein progressive tool call chunkstext/imageUrlfields when OCI streams TEXT or IMAGE content itemsOnly the tool call padding survived the merge. The content item padding (6 lines) was silently lost because upstream's Oracle PR (
61fed95f8c) added similar-looking tool call code but NOT content item handling. The resolver kept upstream's version, not realizing CARTO's was a superset.Why the pipeline missed it
def/class/async defdefinitions. The content item padding is inline code inside an existing function — invisible to pattern matching.Impact
Deployed image caused Pydantic validation errors on CartoDB/cloud-native#23162 when OCI streamed TEXT chunks without a
textfield.Fix
Cherry-picked
9505d65759ontoupstream-sync/v1.81.0-stableto restore the content item padding. Image rebuild required.🔧 CARTO Feature Fixes Applied
Status: ✅ Fixed
Features Restored: 1
PR #90: feat(docker): multi-arch builds (AMD64 + ARM64) for non_root image
Fixed: 2026-02-11 02:13:19 UTC
Workflow Run: #14
CARTO Customizations Analysis
Overall Assessment: ✅ PASS
CARTO Feature Preservation Analysis
Summary
Overall Assessment: PASS
All CARTO customizations have been successfully preserved or properly integrated during the upstream sync to v1.81.0-stable. The
.github/carto-features.ymlmanifest verification confirms all critical features are present.Feature Details
Preserved CARTO (8 features)
These features have no upstream equivalent and were fully preserved:
/chat/completions,/embeddingsetc. from deployment URLs to prevent 404sJSONDecoder.raw_decode()to fix malformed JSON from Gemini streaming_handle_tool_call_delta()SnowflakeStreamingHandlerfor missing 'created' field and tool_use transformation_transform_messages(),_transform_tool_choice(), default tool_choice_store_session_in_redis()for streaming conversation contextdocker-build-multiarch.yamlwith AMD64 + ARM64 native buildsCustomized Upstream (4 features)
These features were merged with upstream improvements:
.get()patternUpstream Substitutes (1 feature)
Verification Status
All features from
.github/carto-features.ymlmanifest verified as present:Issues Found
None. All CARTO customizations were properly handled during the sync.
Key Restoration Commits
The following commits restored CARTO features that were initially dropped during the merge:
e5a989447c3a56fd3fb7c3c2f0c29fe48b6fee80963cbRecommendations
Analysis completed: 2026-02-11
Feature-by-Feature Breakdown
PR #70: fix(azure): Strip operation suffixes from deployment URLs
PR #54: fix: repair malformed JSON in streaming tool call arguments
PR #68: fix(oci): Add tool calling support for OCI Gemini streaming
PR #38: fix: Snowflake PAT auth and Claude streaming support
PR #58: fix: Enable Snowflake tool calling via Responses API
PR #69: fix(databricks): Handle empty delta in GPT-5 streaming responses
PR #5: fix: Redis session storage for streaming Response API requests
PR #90: feat(docker): multi-arch builds (AMD64 + ARM64) for non_root image
PR #55: fix: switch Docker base image to wolfi-base for Python version control
PR #13: Download prisma binaries on build time
PR #61: fix: Azure Responses API URL construction with deployment paths
PR #66: fix(responses): Fix _ENTERPRISE_ResponsesSessionHandler NameError
Analyzed: 2026-02-11 02:24:36 UTC
Workflow Run: #15
Analysis Artifacts: Download JSON/MD
Method: Claude Code (Opus 4.5) post-resolution semantic analysis