ci: dump debug data about builds in CI

[brettlangdon]

Helper to collect debug metadata about our extension building.

This helper will collect timing information for the overall (mostly) build execution, along with per-extension timings, and then dump the data to disk when complete.

This is only enabled when _DD_DEBUG_EXT env variable is set (to anything).

It'll dump to the file debug_ext_metadata.txt with the format:

Environment:
	DD_COMPILE_DEBUG: False
	DD_USE_SCCACHE: False
	CMAKE_BUILD_PARALLEL_LEVEL: 12
Total time: 154.28s
Extension build times:
	Total: 147.76s (95.77%)
	ddtrace.appsec._iast._taint_tracking._native: 37.13s (24.07%)
	ddtrace.internal.datadog.profiling.ddup._ddup: 28.21s (18.28%)
	ddtrace.internal.core._core: 15.81s (10.24%)
	ddtrace.profiling.exporter.pprof: 13.25s (8.59%)
	ddtrace.internal.datadog.profiling.stack_v2._stack_v2: 12.76s (8.27%)
	ddtrace.internal._encoding: 10.24s (6.64%)
	ddtrace.profiling.collector.stack: 8.98s (5.82%)
	ddtrace.internal.datadog.profiling.crashtracker._crashtracker: 7.62s (4.94%)
	ddtrace.internal._tagset: 3.07s (1.99%)
	ddtrace.profiling.collector._task: 2.94s (1.91%)
	ddtrace.profiling._threading: 2.45s (1.58%)
	ddtrace.profiling.collector._traceback: 1.66s (1.08%)
	ddtrace.internal._rand: 1.26s (0.82%)
	ddtrace.internal._threads: 0.76s (0.49%)
	ddtrace.profiling.collector._memalloc: 0.48s (0.31%)
	ddtrace.vendor.psutil._psutil_linux: 0.47s (0.30%)
	ddtrace.profiling._build: 0.29s (0.19%)
	ddtrace.vendor.psutil._psutil_posix: 0.26s (0.17%)
	ddtrace.appsec._iast._stacktrace: 0.11s (0.07%)

This is enabled in our GitLab build_base_venvs job, and the debug_ext_metadata.txt file is saved as an artifact.

As well, a tiny/easy improvement added was to mirror the CMake parallel env variable with CARGO_BUILD_JOBS as well. (I can move this to another PR if desired)

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[brettlangdon]

I want to add CARGO_BUILD_JOBS to the metadata output.

done

[github-actions]

CODEOWNERS have been resolved as:

.gitignore                                                              @DataDog/apm-core-python
.gitlab/tests.yml                                                       @DataDog/python-guild @DataDog/apm-core-python
hatch.toml                                                              @DataDog/python-guild
riotfile.py                                                             @DataDog/apm-python
scripts/docs/build.sh                                                   @DataDog/python-guild
setup.py                                                                @DataDog/python-guild

[sanchda]

This is awesome, nice!

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: brettlangdon/debug.ext.builds
Commit report: 34eb5a7
Test service: dd-trace-py

✅ 0 Failed, 863 Passed, 605 Skipped, 15m 20.59s Total duration (20m 40.56s time saved)

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-12-20 19:53:47

Comparing candidate commit 34eb5a7 in PR branch brettlangdon/debug.ext.builds with baseline commit 6f3bf25 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics.