Skip to content

[ISSUE] get_langchain_chat_open_ai_client() fails with 401 on async operations (ainvoke, astream) #1173

@mr-brobot

Description

@mr-brobot

get_langchain_chat_open_ai_client() only configures the synchronous HTTP client with Databricks authentication. Async operations (ainvoke(), astream(), abatch()) use an unauthenticated default client and fail with HTTP 401 Unauthorized.

The method passes http_client but not http_async_client to ChatOpenAI:

# databricks/sdk/mixins/open_ai_client.py:105-110
return ChatOpenAI(
    model=model,
    openai_api_base=self._api._cfg.host + "/serving-endpoints",
    api_key="no-token",
    http_client=self._get_authorized_http_client(),
    # http_async_client is NOT set
)

LangChain maintains separate HTTP clients for sync and async operations. When http_async_client is unset, it creates a default httpx.AsyncClient. Since Databricks authentication is injected via BearerAuth on the httpx client, the default async client lacks the middleware that actually authenticates requests.

Reproduction

import asyncio
from langchain_core.messages import HumanMessage
from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
llm = w.serving_endpoints.get_langchain_chat_open_ai_client(model='my-endpoint')

# Sync works
llm.invoke([HumanMessage(content='ping')])

# Async fails with 401
asyncio.run(llm.ainvoke([HumanMessage(content='ping')]))

Expected behavior

Both sync and async operations should authenticate identically and succeed.

Is it a regression?

No. This has been the behavior since get_langchain_chat_open_ai_client() was introduced in PR #779.

Other Information

  • OS: Any
  • Version: 0.76.0 (and all prior versions with this method)
  • langchain-openai: 1.1.6

Additional context

This blocks all async LangChain patterns with Databricks model serving:

  • LangGraph agents (create_react_agent() uses ainvoke() internally)
  • Async streaming (astream(), astream_events())
  • Batch async (abatch())

Proposed fix: Add http_async_client parameter. The existing BearerAuth class already supports async via the generator-based auth_flow() pattern:

def _get_authorized_async_http_client(self):
    import httpx
    databricks_token_auth = BearerAuth(self._api._cfg.authenticate)
    return httpx.AsyncClient(auth=databricks_token_auth)

Workaround: Manually construct ChatOpenAI with both clients:

import httpx
from langchain_openai import ChatOpenAI

class BearerAuth(httpx.Auth):
    def __init__(self, fn):
        self._fn = fn
    def auth_flow(self, request):
        request.headers["Authorization"] = self._fn()["Authorization"]
        yield request

w = WorkspaceClient()
auth = BearerAuth(w.config.authenticate)

llm = ChatOpenAI(
    model="my-endpoint",
    openai_api_base=f"{w.config.host}/serving-endpoints",
    api_key="no-token",
    http_client=httpx.Client(auth=auth),
    http_async_client=httpx.AsyncClient(auth=auth),
)

Related: #847 (same root cause for get_open_ai_client())

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions