[BUG]: AsyncGenerator spans created via OpenTelemetry API lose parent-child relationship with ddtrace native instrumentation #16261
Description
Tracer Version(s)
3.15.0
Python Version(s)
3.13.x
Pip Version(s)
uv 0.7.12
Bug Report
Summary
When using OpenTelemetry API to create spans for AsyncGenerator functions with DD_TRACE_OTEL_ENABLED=true, the parent-child relationship is broken between
OTel-created spans and ddtrace native instrumentation spans (e.g., OpenAI, httpx).
Environment
- ddtrace version: 3.15.0
- Python version: 3.13
- opentelemetry-api version: 1.37.0
- opentelemetry-sdk version: 1.37.0
- DD_TRACE_OTEL_ENABLED: true
Root Cause Analysis
-
OTel spans are created via
opentelemetry.trace.get_tracer().start_as_current_span(), which stores context in OpenTelemetry'scontextvars -
ddtrace native instrumentation (e.g., OpenAI auto-instrumentation in
ddtrace/contrib/internal/openai/patch.py) usespin.tracer.trace()directly, which reads
parent span from_DD_CONTEXTVAR -
DDRuntimeContext.attach()synchronizes OTel context to ddtrace context whenuse_span()is called -
The issue: During AsyncGenerator iteration, after
yield, the synchronization between OTel context and_DD_CONTEXTVARappears to be incomplete. When ddtrace
native instrumentation creates a span, it reads from_DD_CONTEXTVARwhich doesn't reflect the current OTel context.
Additional Context
- Works correctly with pure OpenTelemetry (e.g., exporting to Jaeger) - parent-child relationship is maintained
- Only occurs when combining OTel API spans + ddtrace native instrumentation within AsyncGenerator context
- The issue affects any ddtrace auto-instrumentation that uses
pin.tracer.trace()internally (OpenAI, httpx, etc.)
Workaround
Currently, we can work around this by using ddtrace native API directly instead of OpenTelemetry API, but this defeats the purpose of DD_TRACE_OTEL_ENABLED.
Reproduction Code
import asyncio
import functools
import inspect
import os
from collections.abc import AsyncGenerator
from typing import Any
from openai import AsyncAzureOpenAI
from opentelemetry import trace
tracer = trace.get_tracer(__name__)
def trace_span(name: str):
"""OTel-based trace decorator with AsyncGenerator support."""
def decorator(func):
if inspect.isasyncgenfunction(func):
@functools.wraps(func)
async def wrapper(*args: Any, **kwargs: Any) -> AsyncGenerator[Any, None]:
with tracer.start_as_current_span(name):
async for item in func(*args, **kwargs):
yield item
return wrapper
return func
return decorator
client = AsyncAzureOpenAI(
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
api_version="2024-12-01-preview",
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
)
@trace_span("myapp.handle_request")
async def handle_request() -> None:
"""Top-level request handler (normal async function)."""
async for chunk in stream_chat():
print(chunk, end="", flush=True)
print()
@trace_span("myapp.stream_chat")
async def stream_chat() -> AsyncGenerator[str, None]:
"""
Stream chat completion (AsyncGenerator function).
This span loses its parent when viewed in Datadog.
"""
response = await client.chat.completions.create(
model="gpt-5",
messages=[{"role": "user", "content": "Say hello in one word."}],
stream=True,
)
async for chunk in response:
if chunk.choices and chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
async def main():
await handle_request()
if __name__ == "__main__":
asyncio.run(main())Run with:
DD_TRACE_OTEL_ENABLED=true ddtrace-run python repro.pyExpected Behavior
myapp.handle_request
└─► myapp.stream_chat
└─► createChatCompletion
└─► http.request
All spans should be connected in a single trace with proper parent-child relationships.
Actual Behavior
myapp.handle_request (trace 1)
myapp.stream_chat (trace 2, orphaned, parent_id=0)
└─► createChatCompletion
└─► http.request
The myapp.stream_chat span (created inside an AsyncGenerator wrapper) becomes an orphaned root span with parent_id=0, breaking away from its parent
myapp.handle_request.
The child spans under myapp.stream_chat (like createChatCompletion from ddtrace auto-instrumentation) remain connected to it, but the entire subtree is disconnected
from the original trace.
Error Logs
No response
Libraries in Use
No response
Operating System
No response