Support prompt caching in bedrock API (@anthropic-ai/bedrock-sdk)

[fcOSENSE]

https://github.com/anthropics/anthropic-sdk-typescript/blob/main/tests/api-resources/beta/prompt-caching/messages.test.ts
I can not use this sample code by using @anthropic-ai/bedrock-sdk package, so does this package will update ?

[RobertCraigie]

The Bedrock API does not yet support prompt caching. The SDK will be updated when it does.

[FrancescoSaverioZuppichini]

The Bedrock API does not yet support prompt caching. The SDK will be updated when it does.

ETA? I mean you guys collaborate with them was wondering if you know something

[RobertCraigie]

Unfortunately I don't have an ETA to share. I'll re-open this issue so you can be notified when support is added!

[willblaschko]

Is the issue that AWS doesn't support it yet?

[bharathvaj-ganesan]

It would be great if AWS bedrock supports this soon. I believe this will significantly reduce the cost.

[FrancescoSaverioZuppichini]

It would be great if AWS bedrock supports this soon. I believe this will significantly reduce the cost.

if you can, use anthropic diretly no real advantage on using bedrock or aws stuff

[stuartburrellfs]

For many organisations (such as ours), there is a huge benefit to AWS - since services in AWS have existing legal/security approval due to other usage core to the business.

Getting independent approval to send company data to the anthropic API would take months.

[FrancescoSaverioZuppichini]

For many organisations (such as ours), there is a huge benefit to AWS - since services in AWS have existing legal/security approval due to other usage core to the business.

Getting independent approval to send company data to the anthropic API would take months.

same for me, this is why big organisations (like ours) cannot really move fast but it is what it is

[bruno-oliveira]

Hello,
Any updates on this? This just became more relevant since Contextual Retrieval was presented.

[hoanghd218]

hello, any update on this? we really need it.

[ojcobb]

Would also appreciate an update/rough ETA

[jasongilman]

Since people are following this to find out when this feature would be available. I thought I'd throw in some suggestions based on feedback from someone in AWS:

• Prompt caching is in beta right now. AWS is unlikely to integrate it until it moves out of that for Anthropic.
• Follow https://aws.amazon.com/about-aws/whats-new/2024/?whats-new-content-all.sort-by=item.additionalFields.postDateTime&whats-new-content-all.sort-order=desc&awsf.whats-new-categories=*all for the latest updates

[j10sanders]

Also important for my company

[ReMuSoMeGA93]

Yeah, can we get an update please?

Even though the issue cannot be resolved by this repo?

I know that this only depends on Bedrock supporting a more recent version of 3.5 Sonnet, since the current version is claude-3-5-sonnet-20240620 (June 20). But still -- maybe the best place for an update isn't AWS -- it's here.

no.

[brianjking]

Bummer that this isn't supported, Bedrock is supporting Prompt Batching - but not caching, 👎

[PhanManhQuyet]

With announce new Claude 3.5 model in aws bedrock, has this feature avaiable in Bedrock?

[brianprost]

Here's to hoping this gets announced at re:invent

[yangcheng]

it's a beta feature in anthropic, not sure if AWS will adopt beta features

[richzw]

Following

[nichaln]

Here's to hoping this gets announced at re:invent

Was just announced over there now 🚀