Access models via Azure APIM

[dminier]

Hello!

First of all, great project!

I was wondering if it's possible to integrate models via Azure APIM, as many enterprises rely on it. You can find simple information on this project here: https://github.com/Azure-Samples/AI-Gateway/tree/main/labs/GPT-4o-inferencing.

My goal is to configure something like this for the llm_endpoint:

elif config.supplier == DefaultModelSuppliers.AZURE_APIM:
    _llm = AzureChatOpenAI(
        azure_deployment=config.deployment,
        api_version=config.api_version,
        azure_endpoint=config.llm_base_url,
        default_headers={
            "Ocp-Apim-Subscription-Key": config.llm_api_key
        },
        default_query=config.default_query,
        max_tokens=config.max_output_tokens,
    )

And for embeddings:

elif settings.azure_apim_openai_embeddings_endpoint:
    logger.debug(f"Using Azure APIM OpenAI embeddings")
    embeddings = AzureOpenAIEmbeddings(
        azure_deployment=settings.azure_apim_openai_embeddings_deployment,
        azure_endpoint=settings.azure_apim_openai_embeddings_endpoint,
        api_version=settings.azure_apim_openai_embeddings_api_version,
        default_headers={
            "Ocp-Apim-Subscription-Key": settings.azure_apim_openai_embeddings_api_key
        },
        default_query=settings.azure_apim_openai_embeddings_default_query,
    )

I'm waiting for Cohere Reranker integration.

In my case, the Azure APIM has a complex URL structure, making it difficult to extract properties directly from the URL. Additionally, we have some extra properties like default_query that need to be handled.

I’ve started working on this here: https://github.com/QuivrHQ/quivr/compare/main...dminier:quivr:feature/azure-apim?expand=1. I've tested it, and it seems to work well.

Thanks!