[Feature]: Context Caching for Vertex AI

[DreamGenX]

The Feature

It looks like Gemini context caching does not work when using Vertex AI.
Doing a cursory search, it looks like this part would need to be implemented to support Vertex AI:

litellm/litellm/llms/vertex_ai_and_google_ai_studio/gemini/transformation.py

Line 392 in c73ce95

await context_caching_endpoints.async_check_and_create_cache(

Motivation, pitch

Context cachin is already supported using the Gemini API and it's a good way to reduce costs.

Twitter / LinkedIn details

No response

[codenprogressive]

+1
Any updates on enabling context caching for LLMs (Gemini, Claude, etc) available through VertexAI?

[krrishdholakia]

@codenprogressive just tested claude on vertex ai - it doesn't look like prompt caching is available there yet.

I plan on picking up vertex ai context caching this week

[codenprogressive]

Hey @krrishdholakia, indeed Claude on Vertex still doesn't support prompt caching! (there is a feature request: anthropics/anthropic-sdk-python#653)

I think for now we can enable it for Gemini on VertexAI.

[emerzon]

AWS has started a preview for the prompt caching for Claude: https://pages.awscloud.com/promptcaching-Preview.html
Hope Vertex comes up next