OpenRouter models usage tracking (caching, cost...)

[ubernion]

Hey,

Im using langfuse through the observe decorator and the openai sdk.

Im calling OpenRouter through their API, and the OpenRouter API provides with each response token usage, total cost, number of cached tokens, savings made thx to cached token etc...

But even tho i followed the instructions from ur docs it doesn't track all that! Do i really have to use a custom model rule, and does thar even help me track share of cached token?

Thx

[dosubot[bot]]

To track additional metrics like cached tokens and cost savings from OpenRouter using Langfuse, you might need to extend the default functionality provided by the observe decorator. The Langfuse documentation does not explicitly mention tracking cached tokens and cost savings, so you may need to manually update the observation with this additional context.

You can use the @observe(as_type="generation") decorator to create generation spans and then update the observation with the additional metrics provided by OpenRouter's API response. This can be done using langfuse_context.update_current_observation() to include details like cached tokens and cost savings.

If the default setup does not meet your needs, implementing a custom model rule might be necessary to capture these specific metrics. However, the documentation does not provide detailed steps for this, so you may need to experiment with the SDK to achieve the desired tracking.

For more detailed guidance, you might want to refer to the Langfuse documentation or explore the SDK's capabilities further ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[thiswillbeyourgithub]

Hi everyone. I made a script to automatically continuously load all the openrouter models pricing into langfuse. It's hosted in this repo: https://github.com/thiswillbeyourgithub/openrouter_cost_into_langfuse/

[clemra]

This is super cool, thank you!