Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: integration with dd-trace-api #12057

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
eb66781
stub integration with dd-trace-api
emmettbutler Jan 22, 2025
198aaa9
a basic but valid failing test
emmettbutler Jan 23, 2025
d60b73c
fleshing out the test setup
emmettbutler Jan 24, 2025
4e1004d
working on the dummy tracer...
emmettbutler Jan 24, 2025
dae3efd
store tracer globally
emmettbutler Jan 24, 2025
3e40d48
pull from github
emmettbutler Jan 24, 2025
181b1c1
add dd_trace_api integration to CI
emmettbutler Jan 24, 2025
ffc79e5
linting
emmettbutler Jan 24, 2025
89e04a1
enable snapshots
emmettbutler Jan 24, 2025
9bc3423
doesnt work on 3.7
emmettbutler Jan 24, 2025
60ca763
stub verification
emmettbutler Jan 24, 2025
4ffec90
stub verification
emmettbutler Jan 24, 2025
b1db4dc
only hook once
emmettbutler Jan 24, 2025
d1a7353
more generic code
emmettbutler Jan 24, 2025
1b3cec2
methods on real spans to avoid bookkeeping
emmettbutler Jan 27, 2025
d3a6196
current_span APIs
emmettbutler Jan 27, 2025
026c06d
use singleton tracer
emmettbutler Jan 27, 2025
e1f1a68
convert stubs to real spans when passed to API functions
emmettbutler Jan 27, 2025
36cdb43
slight refactors
emmettbutler Jan 27, 2025
9bba449
remove some duplication from handlers list
emmettbutler Jan 27, 2025
2448b3e
remove some duplication from handlers list
emmettbutler Jan 27, 2025
35d8a49
reno
emmettbutler Jan 27, 2025
66bdb34
Merge branch 'main' into emmett.butler/dd-trace-api-integration
emmettbutler Jan 27, 2025
aaedab9
chore: update changelog for version 2.19.2 (#12088)
Yun-Kim Jan 27, 2025
e3045a1
fix(profiling): fix SystemError when collecting memory profiler event…
nsrip-dd Jan 27, 2025
55767a7
chore(tracing): refactor web server integrations to use the core modu…
wconti27 Jan 28, 2025
16d5280
ci(tracer): make serverless test unrequired (#12121)
christophe-papazian Jan 28, 2025
4f0bcb5
chore(asm): clean libddwaf loading (#12102)
christophe-papazian Jan 28, 2025
b787857
simplify
emmettbutler Jan 28, 2025
86d69dd
simplify
emmettbutler Jan 28, 2025
bae4cac
unused import
emmettbutler Jan 28, 2025
ee4492c
some more tests
emmettbutler Jan 28, 2025
c4448ea
fix(llmobs): propagate distributed headers via signal dispatching, no…
Yun-Kim Jan 28, 2025
cb41f8e
feat(provider): expose context provider in ddtrace.trace (#12135)
mabdinur Jan 29, 2025
50ce3bd
Merge branch 'main' into emmett.butler/dd-trace-api-integration
emmettbutler Jan 29, 2025
af9098c
chore(ci): skip non-linux OCI package creation (#12036)
randomanderson Jan 30, 2025
abca08f
Merge branch 'main' into emmett.butler/dd-trace-api-integration
emmettbutler Jan 31, 2025
17552de
add patching for tracer.wrap, which requires API accessing the return…
emmettbutler Feb 3, 2025
d147067
remove tests for removed functionality
emmettbutler Feb 3, 2025
db87343
remove tests
emmettbutler Feb 4, 2025
e043f22
code organization
emmettbutler Feb 4, 2025
5540bd1
no shared state
emmettbutler Feb 5, 2025
acac914
update requirements
emmettbutler Feb 5, 2025
0e679f8
one hook name
emmettbutler Feb 5, 2025
09c32e3
Merge branch 'main' into emmett.butler/dd-trace-api-integration
emmettbutler Feb 6, 2025
147891b
merge conflict
emmettbutler Feb 6, 2025
8620f4c
merge conflict
emmettbutler Feb 6, 2025
ff23412
disallow configuring dd_trace_api patching via envvars
emmettbutler Feb 6, 2025
d01d844
update suitespec to match reality
emmettbutler Feb 6, 2025
a2a4831
update import that broke with major version
emmettbutler Feb 6, 2025
2f07b12
change order of conditions to avoid a few instructions
emmettbutler Feb 7, 2025
94a7494
simpler span proxying code
emmettbutler Feb 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .riot/requirements/10e65d1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/10e65d1.in
#
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py
exceptiongroup==1.2.2
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
mock==5.1.0
opentracing==2.4.0
packaging==24.2
pluggy==1.5.0
pytest==8.3.4
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
sortedcontainers==2.4.0
tomli==2.2.1
urllib3==2.3.0
25 changes: 25 additions & 0 deletions .riot/requirements/1261872.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/1261872.in
#
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
mock==5.1.0
opentracing==2.4.0
packaging==24.2
pluggy==1.5.0
pytest==8.3.4
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
sortedcontainers==2.4.0
urllib3==2.3.0
27 changes: 27 additions & 0 deletions .riot/requirements/14d1688.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#
# This file is autogenerated by pip-compile with Python 3.8
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/14d1688.in
#
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.1
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py
exceptiongroup==1.2.2
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
mock==5.1.0
opentracing==2.4.0
packaging==24.2
pluggy==1.5.0
pytest==8.3.4
pytest-cov==5.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
sortedcontainers==2.4.0
tomli==2.2.1
urllib3==2.2.3
27 changes: 27 additions & 0 deletions .riot/requirements/668f2f5.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#
# This file is autogenerated by pip-compile with Python 3.9
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/668f2f5.in
#
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py
exceptiongroup==1.2.2
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
mock==5.1.0
opentracing==2.4.0
packaging==24.2
pluggy==1.5.0
pytest==8.3.4
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
sortedcontainers==2.4.0
tomli==2.2.1
urllib3==2.3.0
25 changes: 25 additions & 0 deletions .riot/requirements/d5f777e.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#
# This file is autogenerated by pip-compile with Python 3.13
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/d5f777e.in
#
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
mock==5.1.0
opentracing==2.4.0
packaging==24.2
pluggy==1.5.0
pytest==8.3.4
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
sortedcontainers==2.4.0
urllib3==2.3.0
25 changes: 25 additions & 0 deletions .riot/requirements/e49670c.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/e49670c.in
#
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
mock==5.1.0
opentracing==2.4.0
packaging==24.2
pluggy==1.5.0
pytest==8.3.4
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
sortedcontainers==2.4.0
urllib3==2.3.0
1 change: 1 addition & 0 deletions ddtrace/_monkey.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
"cassandra": True,
"celery": True,
"consul": True,
"dd_trace_api": True,
"django": True,
"dramatiq": True,
"elasticsearch": True,
Expand Down
23 changes: 23 additions & 0 deletions ddtrace/contrib/dd_trace_api/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
"""

Enabling
emmettbutler marked this conversation as resolved.
Show resolved Hide resolved
~~~~~~~~

The dd_trace_api integration is enabled automatically when using
:ref:`ddtrace-run<ddtracerun>` or :ref:`import ddtrace.auto<ddtraceauto>`.

Or use :func:`patch()<ddtrace.patch>` to manually enable the integration::

from ddtrace import patch
patch(dd_trace_api=True)



Global Configuration
~~~~~~~~~~~~~~~~~~~~


Instance Configuration
~~~~~~~~~~~~~~~~~~~~~~

"""
Empty file.
73 changes: 73 additions & 0 deletions ddtrace/contrib/internal/dd_trace_api/patch.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
from sys import addaudithook

import dd_trace_api

import ddtrace


_DD_HOOK_PREFIX = "dd.hooks."
_CURRENT_SPAN = None
_TRACER = ddtrace.tracer


def _patched_start_span(*args, **kwargs):
global _CURRENT_SPAN
_CURRENT_SPAN = _TRACER.start_span(*args, **kwargs)


def _patched_trace(*args, **kwargs):
global _CURRENT_SPAN
_CURRENT_SPAN = _TRACER.trace(*args, **kwargs)


def _patched_span_enter(*args, **kwargs):
_CURRENT_SPAN.__enter__(*args, **kwargs)


def _patched_span_exit(*args, **kwargs):
_CURRENT_SPAN.__exit__(*args, **kwargs)


def _patched_span_finish(*args, **kwargs):
_CURRENT_SPAN.finish(*args, **kwargs)


_HANDLERS = {
"Tracer.start_span": _patched_start_span,
"Tracer.trace": _patched_trace,
"Span.__enter__": _patched_span_enter,
"Span.__exit__": _patched_span_exit,
"Span.finish": _patched_span_finish,
}


def _hook(name, args):
if not dd_trace_api.__datadog_patch:
return
if name.startswith(_DD_HOOK_PREFIX):
name_suffix = ".".join(name.split(".")[2:])
if name_suffix not in _HANDLERS:
return
_HANDLERS[name_suffix](*(args[0][0]), **(args[0][1]))


def get_version() -> str:
return getattr(dd_trace_api, "__version__", "")


def patch(tracer=None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure the integration patch/unpatch approach makes sense here, should we always register an audit hook that listens for these events?

are there ever any cases when we wouldn't want to listen to the events? it would probably be better to have dd_trace_api not emit events if you want to "disable" vs having that handled here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

patch, unpatch and get_version are here just to make this code work with the existing contrib structure. The contrib is enabled by default, and it can be disabled by the same method as any other default-on contrib. As far as I can tell, this code makes sense as a contrib because that provides lazy loading.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, the lazy loading makes sense, but I am not sure we should provide all the same functionality, like DD_DD_TRACE_API_ENABLED=false or DD_PATCH_MODULES=dd_trace_api:false or patch_all(dd_trace_api=False), etc (or w/e all those options are).

lazy loading, reporting usage and the version to telemetry both definitely make sense.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add some code that disables those options for this contrib

Copy link
Collaborator Author

@emmettbutler emmettbutler Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ff23412 takes care of disabling via environment variable. patch_all is going to be deprecated, so we don't need to disallow dd_trace_api's use there. #11918

if getattr(dd_trace_api, "__datadog_patch", False):
return
dd_trace_api.__datadog_patch = True
global _TRACER
_TRACER = tracer
if not getattr(dd_trace_api, "__dd_has_audit_hook", False):
addaudithook(_hook)
emmettbutler marked this conversation as resolved.
Show resolved Hide resolved
dd_trace_api.__dd_has_audit_hook = True


def unpatch():
if not getattr(dd_trace_api, "__datadog_patch", False):
return
dd_trace_api.__datadog_patch = False
# NB sys.addaudithook's cannot be removed
14 changes: 8 additions & 6 deletions lib-injection/sources/min_compatible_versions.csv
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,15 @@ algoliasearch,~=2.5
anthropic,==0.26.0
anyio,>=3.4.0
aredis,0
asgiref,~=3.0
asgiref,~=3.0.0
astunparse,0
async_generator,~=1.10
asyncpg,~=0.23
asyncpg,~=0.23.0
asynctest,==0.13.0
austin-python,~=1.0
avro,0
azure.functions,0
bcrypt,==4.2.1
blinker,0
boto3,==1.34.49
bottle,>=0.12
Expand Down Expand Up @@ -64,19 +65,20 @@ elasticsearch[async],0
envier,==0.5.2
exceptiongroup,0
faiss-cpu,==1.8.0
falcon,~=3.0
falcon,~=3.0.0
fastapi,~=0.64.0
flask,~=0.12.0
flask-caching,~=1.10.0
flask-openapi3,0
gevent,~=20.12.0
git+https://github.com/DataDog/dd-trace-api-py,0
google-ai-generativelanguage,0
google-generativeai,0
googleapis-common-protos,0
graphene,~=3.0.0
graphql-core,~=3.2.0
graphql-relay,0
greenlet,~=1.0.0
greenlet,~=1.0
grpcio,~=1.34.0
gunicorn,==20.0.4
gunicorn[gevent],0
Expand All @@ -97,13 +99,13 @@ langchain-community,==0.0.14
langchain-core,==0.1.52
langchain-openai,==0.1.6
langchain-pinecone,==0.1.0
langchain_experimental,==0.0.47
langgraph,~=0.2.60
logbook,~=1.0.0
loguru,~=0.4.0
lxml,0
lz4,0
mako,~=1.1.0
mariadb,~=1.0.0
mariadb,~=1.0
markupsafe,<2.0
mock,0
molten,>=1.0
Expand Down
14 changes: 8 additions & 6 deletions min_compatible_versions.csv
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,15 @@ algoliasearch,~=2.5
anthropic,==0.26.0
anyio,>=3.4.0
aredis,0
asgiref,~=3.0
asgiref,~=3.0.0
astunparse,0
async_generator,~=1.10
asyncpg,~=0.23
asyncpg,~=0.23.0
asynctest,==0.13.0
austin-python,~=1.0
avro,0
azure.functions,0
bcrypt,==4.2.1
blinker,0
boto3,==1.34.49
bottle,>=0.12
Expand Down Expand Up @@ -64,19 +65,20 @@ elasticsearch[async],0
envier,==0.5.2
exceptiongroup,0
faiss-cpu,==1.8.0
falcon,~=3.0
falcon,~=3.0.0
fastapi,~=0.64.0
flask,~=0.12.0
flask-caching,~=1.10.0
flask-openapi3,0
gevent,~=20.12.0
git+https://github.com/DataDog/dd-trace-api-py,0
google-ai-generativelanguage,0
google-generativeai,0
googleapis-common-protos,0
graphene,~=3.0.0
graphql-core,~=3.2.0
graphql-relay,0
greenlet,~=1.0.0
greenlet,~=1.0
grpcio,~=1.34.0
gunicorn,==20.0.4
gunicorn[gevent],0
Expand All @@ -97,13 +99,13 @@ langchain-community,==0.0.14
langchain-core,==0.1.52
langchain-openai,==0.1.6
langchain-pinecone,==0.1.0
langchain_experimental,==0.0.47
langgraph,~=0.2.60
logbook,~=1.0.0
loguru,~=0.4.0
lxml,0
lz4,0
mako,~=1.1.0
mariadb,~=1.0.0
mariadb,~=1.0
markupsafe,<2.0
mock,0
molten,>=1.0
Expand Down
6 changes: 6 additions & 0 deletions riotfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -756,6 +756,12 @@ def select_pys(min_version: str = MIN_PYTHON_VERSION, max_version: str = MAX_PYT
),
],
),
Venv(
name="dd_trace_api",
command="pytest {cmdargs} tests/contrib/dd_trace_api",
pkgs={"git+https://github.com/DataDog/dd-trace-api-py": latest, "requests": latest},
pys=select_pys(min_version="3.8"),
),
# Django Python version support
# 2.2 3.5, 3.6, 3.7, 3.8 3.9
# 3.2 3.6, 3.7, 3.8, 3.9, 3.10
Expand Down
Empty file.
Loading
Loading