-
Notifications
You must be signed in to change notification settings - Fork 482
Open
Labels
Description
Tracer Version(s)
4.3.2
Python Version(s)
3.14.2
Pip Version(s)
uv 0.9.8
Bug Report
The PydanticAI LLMObs integration extracts user input solely from the user_prompt argument (first positional arg to Agent.run()/Agent.iter()). When messages are passed via message_history instead, INPUT_VALUE on the agent span is None.
This affects PydanticAI's VercelAIAdapter (and any other caller that uses message_history instead of user_prompt). The adapter converts incoming messages into PydanticAI ModelMessage objects and passes them as message_history — it never sets user_prompt.
Reproduction Code
# /// script
# requires-python = ">=3.12"
# dependencies = [
# "pydantic-ai[openai]>=1.29.0",
# "ddtrace>=4.3.0",
# ]
# ///
"""Repro: ddtrace LLMObs doesn't tag user input when using VercelAIAdapter.
VercelAIAdapter passes all messages via `message_history`, never setting
`user_prompt`. ddtrace's PydanticAI integration only reads `user_prompt`,
so INPUT_VALUE on the agent span is always None.
Requires: OPENAI_API_KEY, DD_API_KEY, DD_SITE
Example: uv run repro_ddtrae_pydantic_input.py
"""
import asyncio
import os
os.environ.setdefault("DD_LLMOBS_AGENTLESS_ENABLED", "1")
os.environ.setdefault("DD_LLMOBS_ML_APP", "ddtrace-pydantic-input-bug-repro")
from ddtrace.llmobs import LLMObs
LLMObs.enable()
# Diagnostic hook: print INPUT_VALUE after ddtrace tags the agent span
from ddtrace.llmobs._constants import INPUT_VALUE
from ddtrace.llmobs._integrations.pydantic_ai import PydanticAIIntegration
_original = PydanticAIIntegration._llmobs_set_tags_agent
def _diagnostic(self, span, args, kwargs, response):
_original(self, span, args, kwargs, response)
print(f"INPUT_VALUE on agent span: {span._get_ctx_item(INPUT_VALUE)!r}")
PydanticAIIntegration._llmobs_set_tags_agent = _diagnostic
from pydantic_ai import Agent
from pydantic_ai.ui.vercel_ai._adapter import VercelAIAdapter
from pydantic_ai.ui.vercel_ai.request_types import SubmitMessage, TextUIPart, UIMessage
agent = Agent("openai:gpt-4o-mini", instructions="Reply in one sentence.")
async def main():
# Simulate a Vercel AI SDK request (as a frontend would send).
# VercelAIAdapter converts this into message_history, never setting user_prompt.
# Expected: INPUT_VALUE = "What is 2 + 2?"
# Actual: INPUT_VALUE = None
request = SubmitMessage(
trigger="submit-message",
id="msg-1",
messages=[
UIMessage(
id="msg-1",
role="user",
parts=[TextUIPart(type="text", text="What is 2 + 2?")],
),
],
)
adapter = VercelAIAdapter(agent, request)
async for event in adapter.run_stream():
pass
LLMObs.flush()
asyncio.run(main())Error Logs
No response
Libraries in Use
- pydantic-ai: 1.29.0
Operating System
Darwin Kernel Version 25.2.0
Reactions are currently unavailable