Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Models/Endpoints/Providers #4922

Open
18 of 23 tasks
krrishdholakia opened this issue Jul 27, 2024 · 22 comments
Open
18 of 23 tasks

New Models/Endpoints/Providers #4922

krrishdholakia opened this issue Jul 27, 2024 · 22 comments

Comments

@krrishdholakia
Copy link
Contributor

krrishdholakia commented Jul 27, 2024

Parent issue to track new models/endpoints/providers to add to litellm, comment below for new ones

@krrishdholakia krrishdholakia pinned this issue Jul 27, 2024
@guleng
Copy link

guleng commented Aug 1, 2024

@krrishdholakia I have deployed the text graph model locally, but I am unable to configure it for use in Litellm. Besides Azure and other vendors' locally deployed text graph models, cannot they be used in Litellm?
image

image
Can you help me with how to configure it?

@seonghobae
Copy link

Please checkout Azure GPT-4o-mini ;) https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/openai-s-gpt-4o-mini-now-available-in-api-with-vision/ba-p/4200640

@guleng
Copy link

guleng commented Aug 1, 2024

@seonghobae Does it not support locally deployed Wenshengtu models? Does it only support models from Azure and OpenAI manufacturers?

@krrishdholakia
Copy link
Contributor Author

krrishdholakia commented Aug 1, 2024

@guleng - your error message from the provider shows a 404 not found error.

If error continues, file a separate issue for this - with config.yaml, steps to repro, and error traceback received.

@lazyhope
Copy link
Contributor

lazyhope commented Aug 3, 2024

Github just released a new inference endpoint: https://models.inference.ai.azure.com, it has free tier for popular models like gpt-4o so would be worth adding to litellm.

Marketplace: https://github.com/marketplace/models

@krrishdholakia
Copy link
Contributor Author

added @lazyhope

@lazyhope
Copy link
Contributor

lazyhope commented Aug 3, 2024

added @lazyhope

Awesome! Thank you!

@yigitkonur
Copy link
Contributor

Hey Llama 3.1 - 405B is added to AWS Bedrock - model ID is meta.llama3-1-405b-instruct-v1:0

I appreciate if you add support and update docs. Thanks!

@krrishdholakia
Copy link
Contributor Author

it's already supported @yigitkonur

"meta.llama3-1-405b-instruct-v1:0": {

"meta.llama3-1-405b-instruct-v1:0",

@simonhir
Copy link

Is there already a way to add embedding models via azure_ai? In specific cohere.embed-multilingual-v3.0
Couldn't find anything in the documentation or code and the cohere provider seams to not allow setting the api_base.

@krrishdholakia
Copy link
Contributor Author

cohere.embed-multilingual-v3.0

that model name looks like the bedrock one @simonhir

Re: azure ai embedding - not yet, can you create an issue, i'll add it tomorrow

@simonhir
Copy link

@krrishdholakia thanks for your fast answer.
I created #5861. Hope that helps.

@AdithyanI
Copy link

@krrishdholakia Google just upgraded the gemini-1.5-pro. There is a new model gemini-1.5-pro-002 in GA. Also the flash. Can we support this please? We are specifically looking for in vertex_ai support. Thank you :)

image

https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemini-1.5-pro-002?hl=en&authuser=1&rapt=AEjHL4O-gl6qi8hhL9DBsAA1wb_wVoUUEkXTkJeARCSgbGPb03zHz3AwlLM98ApP59b3JpuRF68lhNwehNMo6lq0CFkU4lc4AyjAXa-eVDAGfubP1PeTUP4&project=win-compute-407516

@jbellis
Copy link

jbellis commented Sep 25, 2024

Support for gemini embedding models, please. (text-embedding-004 is free!)

@ClancyDennis
Copy link

Are there any plans to add the /image/edits endpoint currently supported by Dalle-2?

@krrishdholakia
Copy link
Contributor Author

Google just upgraded the gemini-1.5-pro. There is a new model gemini-1.5-pro-002 in GA. Also the flash. Can we support this please?

done -

"gemini-1.5-pro-002": {

Support for gemini embedding models, please. (text-embedding-004 is free!)

@jbellis can you create an issue for this? it would help to know if you're v0 is for google ai studio or vertex ai

Are there any plans to add the /image/edits endpoint currently supported by Dalle-2?

@ClancyDennis didn't realize that, please create an issue, so we can track this.

@krrishdholakia krrishdholakia unpinned this issue Oct 3, 2024
@FireballDWF
Copy link

FireballDWF commented Dec 25, 2024

Related to Add Amazon Titan Text Premier #3774

@igorlima
Copy link
Contributor

Hey @jbellis and @krrishdholakia!

  • Support for gemini embedding models, please. (text-embedding-004 is free!) +

  • can you create an issue for this? +

Great news! litellm already has built-in support for Gemini embeddings, though it wasn't highlighted in the documentation until now. 🎉

I've just proposed that with the PR below, which adds Gemini to the documentation page's list of supported embeddings.

code snippet

Here's a quick and easy code snippet to show you how to use Gemini embeddings with litellm:

import os
os.environ["GEMINI_API_KEY"] = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

from litellm import embedding
response = embedding(
  model="gemini/text-embedding-004",
  input=["good morning from litellm"],
)
print(response)

@KamilFatkhiev
Copy link

Hi @krrishdholakia, are you considering adding support for yandexgpt and gigachat family models?

@krrishdholakia
Copy link
Contributor Author

@KamilFatkhiev
Copy link

@krrishdholakia
Copy link
Contributor Author

@KamilFatkhiev i don't have access to it - but if there's a mock endpoint we can use for testing it should be possible to add it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests