Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Azure OpenAI to check ModelAuthProvider during Embedding and Image model creation #1162

Merged

Conversation

sberyozkin
Copy link
Contributor

@sberyozkin sberyozkin commented Dec 14, 2024

Fixes #1155.

@geoand I've updated it the same way we did checks for ChatModel and StreamingChatModel.

I've confirmed that with this PR, after deploying text-embedding-ada-002 in my Azure OpenAI account, with the secure-poem-multiple-providers demo, that if I add

quarkus.langchain4j.openai.embedding-model.provider=azure-openai
quarkus.langchain4j.azure-openai.embedding-model.resource-name=quarkus-langchain4j
#default deployment name
quarkus.langchain4j.azure-openai.embedding-model.deployment-name=text-embedding-ada-002
quarkus.langchain4j.azure-openai.embedding-model.log-requests=true
quarkus.langchain4j.azure-openai.embedding-model.log-responses=true

and have EmbeddingModel injected and the Azure OpenAI related method updated to

@GET
@Path("azure-openai")
public String getPoemAzureOpenAI() {
    Response<AiMessage> response = azureOpenAI.generate(USER_MESSAGE);
    TokenUsage tokenUsage = embeddingModel.embed(response.content().text()).tokenUsage();
    return response.content().text() + ". Input token: " + tokenUsage.inputTokenCount();
}

I get the token count and logs confirming the remote call was successful, with the user authentication token:

Request:
- method: POST
- url: https://quarkus-langchain4j.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-05-15
- headers: [Accept: application/json], [Authorization: Be...Mw], [Content-Type: application/json], [User-Agent: langchain4j-quarkus-azure-openai], [content-length: 226]
- body: {
  "model" : "text-embedding-ada-002",
  "input" : [ "Java, oh Java, language of code,\nYou make my programs run and decode.\nWith syntax clean and concise,\nYou bring my projects to life.\n\n- Written by the ModelMaster" ]
}

Response:

 headers: [Content-Length: 33488], [Content-Type: application/json], [apim-request-id: ...], [x-ratelimit-remaining-requests: 119], ..., [x-ms-region: West Europe], [azureml-model-session: ...], [x-ratelimit-remaining-tokens: 119959], [Date: Sat, 14 Dec 2024 17:41:07 GMT]
- body: {
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [
        0.0070286347,
        -0.007915613,
        -0.0083591025,
         ...
        -0.018787824
      ]
    }
  ],
  "model": "ada",
  "usage": {
    "prompt_tokens": 36,
    "total_tokens": 36
  }
}

I haven't actually updated the demo, because Google authentication plus Gemini is also shown there and it is not clear to me yet what Vertex-AI offers for embedding models, while having Azure OpenAI only embedding model would impact users who only want to try the demo with Gemini... May be we can do an advanced, everything Azure OpenAI kind of demo later, but for now, I can confirm the updates in this PR are sufficient.

@sberyozkin sberyozkin requested a review from a team as a code owner December 14, 2024 18:07
Copy link
Collaborator

@geoand geoand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

This comment has been minimized.

Copy link

quarkus-bot bot commented Dec 16, 2024

Status for workflow Build (on pull request)

This is the status report for running Build (on pull request) on commit 616d02e.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

@geoand geoand merged commit 22fdfce into quarkiverse:main Dec 16, 2024
65 checks passed
@sberyozkin sberyozkin deleted the azure_openai_embedded_with_model_provider branch December 16, 2024 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Azure OpenAI auth - insist in obsolete api-key when using ModelAuthProvider
2 participants