-
Notifications
You must be signed in to change notification settings - Fork 180
Description
Description
The Config object fails validation when multiple authentication credentials are present in the environment, even when a specific auth method is explicitly configured via parameters.
When explicitly passing client_id and client_secret to the Config object, the SDK still scans environment variables and fails if DATABRICKS_TOKEN is also present, despite not being used.
The explicit auth configuration passed to Config should take precedence over environment variables, or there should be an option (e.g., auth_type) to restrict which credentials are considered.
File "/home/circleci/project/.venv/lib/python3.11/site-packages/databricks/sdk/config.py", line 190, in __init__
raise ValueError(message) from e
ValueError: validate: more than one authorization method configured: oauth and pat. Config: host=https://*****************************************, token=***, client_id=************************************, client_secret=***. Env: DATABRICKS_TOKEN, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRETReproduction
import os
from databricks.sdk import Config
os.environ["DATABRICKS_TOKEN"] = "dapi_xxx"
os.environ["DATABRICKS_CLIENT_ID"] = "my-client-id"
os.environ["DATABRICKS_CLIENT_SECRET"] = "my-client-secret"
# Explicitly configure OAuth, but SDK fails due to DATABRICKS_TOKEN in env
config = Config(
host="https://my-workspace.cloud.databricks.com",
client_id=os.environ["DATABRICKS_CLIENT_ID"],
client_secret=os.environ["DATABRICKS_CLIENT_SECRET"]
)Expected behavior
When auth credentials are explicitly passed to Config, those should take precedence and the SDK should not fail due to unrelated environment variables.
Possible solutions:
- Explicit parameters override environment variables
- Add an
auth_typeparameter that restricts which credentials are considered - Add an
ignore_environmentflag to disable env var scanning
Is it a regression?
Unknown — this may have been the intended behavior since unified auth was introduced.
Debug Logs
N/A — the error occurs during Config.__init__ before debug logging can be captured.
Other Information
- OS: Linux (CircleCI)
- Python: 3.11
- databricks-sdk version: 0.47.0
Additional context
This is particularly problematic in CI/CD environments where multiple Databricks credentials may exist for different purposes (e.g., PAT for one tool, OAuth for another). The current behavior forces users to carefully unset environment variables, which is error-prone and unintuitive.
This issue also affects downstream projects like dbt-databricks, where users configure OAuth in profiles.yml but the adapter fails because an unrelated DATABRICKS_TOKEN exists in the environment.
SImilar issue: #680