Create a flare contents smoke test #9440

MattAlp · 2025-08-29T18:36:22Z

What Does This Do

This adds a series of smoke test cases which validate flare contents, based on agent configuration and reporters.

Motivation

While hacking on #9433, we noticed that there are currently no tests for the flare functionality (or individual reporters). This aims to remedy the lack of coverage and adhere to our Java LP best practices.

Additional Notes

I don't write much Groovy, so the test could likely be more idiomatic in style.

At the moment, this only tests 2 scenarios:

Tracing enabled, profiling enabled (default for any smoke test)
Tracing enabled, profiling disabled

If this is merged it may make sense to refine the expected & optional filename sets, and to get individual component owners to add new test cases covering their reporter behaviors. Note that this checks for the presence of files, rather than the correctness of file contents / reporter implementations.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROF-12402]

datadog-datadog-prod-us1 · 2025-08-29T19:41:04Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 57.60% (-0.04%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 2f8c79e | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-08-29T19:51:22Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mattalp/baseline-flarereporter-smoketesting
git_commit_date	1756820968	1756826481
git_commit_sha	`890497c`	`2f8c79e`
release_version	1.53.0-SNAPSHOT~890497c9aa	1.53.0-SNAPSHOT~2f8c79eb55

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756828334	1756828334
ci_job_id	1109209807	1109209807
ci_pipeline_id	75310380	75310380
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-pa5w5mvu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-pa5w5mvu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 9 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~2f8c79eb55, baseline=1.53.0-SNAPSHOT~890497c9aa

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057285
Total [baseline] (8.631 s) : 0, 8630522
Agent [candidate] (1.045 s) : 0, 1044869
Total [candidate] (8.633 s) : 0, 8632564
section iast
Agent [baseline] (1.178 s) : 0, 1178349
Total [baseline] (9.36 s) : 0, 9360111
Agent [candidate] (1.181 s) : 0, 1181408
Total [candidate] (9.361 s) : 0, 9360604

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.057 s	-
Agent	iast	1.178 s	121.064 ms (11.5%)
Total	tracing	8.631 s	-
Total	iast	9.36 s	729.589 ms (8.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.045 s	-
Agent	iast	1.181 s	136.538 ms (13.1%)
Total	tracing	8.633 s	-
Total	iast	9.361 s	728.041 ms (8.4%)

gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~2f8c79eb55, baseline=1.53.0-SNAPSHOT~890497c9aa

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (741.505 ms) : 0, 741505
BytebuddyAgent [candidate] (731.23 ms) : 0, 731230
GlobalTracer [baseline] (244.889 ms) : 0, 244889
GlobalTracer [candidate] (242.329 ms) : 0, 242329
AppSec [baseline] (30.635 ms) : 0, 30635
AppSec [candidate] (30.017 ms) : 0, 30017
Debugger [baseline] (6.151 ms) : 0, 6151
Debugger [candidate] (6.051 ms) : 0, 6051
Remote Config [baseline] (682.832 µs) : 0, 683
Remote Config [candidate] (662.084 µs) : 0, 662
Telemetry [baseline] (10.786 ms) : 0, 10786
Telemetry [candidate] (12.162 ms) : 0, 12162
section iast
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.46 ms) : 0, 1460
BytebuddyAgent [baseline] (850.979 ms) : 0, 850979
BytebuddyAgent [candidate] (853.014 ms) : 0, 853014
GlobalTracer [baseline] (233.45 ms) : 0, 233450
GlobalTracer [candidate] (233.401 ms) : 0, 233401
AppSec [baseline] (26.816 ms) : 0, 26816
AppSec [candidate] (25.01 ms) : 0, 25010
Debugger [baseline] (5.708 ms) : 0, 5708
Debugger [candidate] (5.762 ms) : 0, 5762
Remote Config [baseline] (597.261 µs) : 0, 597
Remote Config [candidate] (596.918 µs) : 0, 597
Telemetry [baseline] (8.202 ms) : 0, 8202
Telemetry [candidate] (8.276 ms) : 0, 8276
IAST [baseline] (30.162 ms) : 0, 30162
IAST [candidate] (32.872 ms) : 0, 32872

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~2f8c79eb55, baseline=1.53.0-SNAPSHOT~890497c9aa

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1046267
Total [baseline] (10.763 s) : 0, 10763031
Agent [candidate] (1.046 s) : 0, 1045849
Total [candidate] (10.734 s) : 0, 10734078
section appsec
Agent [baseline] (1.229 s) : 0, 1228764
Total [baseline] (10.836 s) : 0, 10835866
Agent [candidate] (1.225 s) : 0, 1224868
Total [candidate] (10.801 s) : 0, 10800915
section iast
Agent [baseline] (1.177 s) : 0, 1176842
Total [baseline] (10.973 s) : 0, 10973120
Agent [candidate] (1.178 s) : 0, 1177602
Total [candidate] (10.911 s) : 0, 10910506
section profiling
Agent [baseline] (1.199 s) : 0, 1198725
Total [baseline] (10.884 s) : 0, 10884056
Agent [candidate] (1.197 s) : 0, 1197481
Total [candidate] (10.881 s) : 0, 10881426

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.229 s	182.497 ms (17.4%)
Agent	iast	1.177 s	130.575 ms (12.5%)
Agent	profiling	1.199 s	152.458 ms (14.6%)
Total	tracing	10.763 s	-
Total	appsec	10.836 s	72.835 ms (0.7%)
Total	iast	10.973 s	210.09 ms (2.0%)
Total	profiling	10.884 s	121.026 ms (1.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.225 s	179.019 ms (17.1%)
Agent	iast	1.178 s	131.753 ms (12.6%)
Agent	profiling	1.197 s	151.632 ms (14.5%)
Total	tracing	10.734 s	-
Total	appsec	10.801 s	66.836 ms (0.6%)
Total	iast	10.911 s	176.428 ms (1.6%)
Total	profiling	10.881 s	147.348 ms (1.4%)

gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~2f8c79eb55, baseline=1.53.0-SNAPSHOT~890497c9aa

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.452 ms) : 0, 1452
crashtracking [candidate] (1.458 ms) : 0, 1458
BytebuddyAgent [baseline] (732.272 ms) : 0, 732272
BytebuddyAgent [candidate] (731.527 ms) : 0, 731527
GlobalTracer [baseline] (241.94 ms) : 0, 241940
GlobalTracer [candidate] (241.96 ms) : 0, 241960
AppSec [baseline] (30.031 ms) : 0, 30031
AppSec [candidate] (30.189 ms) : 0, 30189
Debugger [baseline] (6.011 ms) : 0, 6011
Debugger [candidate] (6.095 ms) : 0, 6095
Remote Config [baseline] (663.128 µs) : 0, 663
Remote Config [candidate] (673.481 µs) : 0, 673
Telemetry [baseline] (12.842 ms) : 0, 12842
Telemetry [candidate] (13.002 ms) : 0, 13002
section appsec
crashtracking [baseline] (1.464 ms) : 0, 1464
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (759.478 ms) : 0, 759478
BytebuddyAgent [candidate] (757.641 ms) : 0, 757641
GlobalTracer [baseline] (236.106 ms) : 0, 236106
GlobalTracer [candidate] (234.943 ms) : 0, 234943
AppSec [baseline] (170.871 ms) : 0, 170871
AppSec [candidate] (170.521 ms) : 0, 170521
Debugger [baseline] (5.756 ms) : 0, 5756
Debugger [candidate] (6.525 ms) : 0, 6525
Remote Config [baseline] (634.727 µs) : 0, 635
Remote Config [candidate] (641.781 µs) : 0, 642
Telemetry [baseline] (9.343 ms) : 0, 9343
Telemetry [candidate] (8.475 ms) : 0, 8475
IAST [baseline] (23.918 ms) : 0, 23918
IAST [candidate] (23.52 ms) : 0, 23520
section iast
crashtracking [baseline] (1.449 ms) : 0, 1449
crashtracking [candidate] (1.443 ms) : 0, 1443
BytebuddyAgent [baseline] (849.05 ms) : 0, 849050
BytebuddyAgent [candidate] (849.686 ms) : 0, 849686
GlobalTracer [baseline] (233.622 ms) : 0, 233622
GlobalTracer [candidate] (232.967 ms) : 0, 232967
AppSec [baseline] (27.612 ms) : 0, 27612
AppSec [candidate] (26.59 ms) : 0, 26590
Debugger [baseline] (5.713 ms) : 0, 5713
Debugger [candidate] (5.756 ms) : 0, 5756
Remote Config [baseline] (596.393 µs) : 0, 596
Remote Config [candidate] (590.846 µs) : 0, 591
Telemetry [baseline] (8.266 ms) : 0, 8266
Telemetry [candidate] (8.314 ms) : 0, 8314
IAST [baseline] (29.46 ms) : 0, 29460
IAST [candidate] (31.296 ms) : 0, 31296
section profiling
crashtracking [baseline] (1.428 ms) : 0, 1428
crashtracking [candidate] (1.431 ms) : 0, 1431
BytebuddyAgent [baseline] (762.195 ms) : 0, 762195
BytebuddyAgent [candidate] (762.746 ms) : 0, 762746
GlobalTracer [baseline] (222.519 ms) : 0, 222519
GlobalTracer [candidate] (222.075 ms) : 0, 222075
AppSec [baseline] (30.135 ms) : 0, 30135
AppSec [candidate] (30.023 ms) : 0, 30023
Debugger [baseline] (6.248 ms) : 0, 6248
Debugger [candidate] (6.288 ms) : 0, 6288
Remote Config [baseline] (714.331 µs) : 0, 714
Remote Config [candidate] (698.065 µs) : 0, 698
Telemetry [baseline] (16.368 ms) : 0, 16368
Telemetry [candidate] (16.429 ms) : 0, 16429
ProfilingAgent [baseline] (108.75 ms) : 0, 108750
ProfilingAgent [candidate] (107.396 ms) : 0, 107396
Profiling [baseline] (109.422 ms) : 0, 109422
Profiling [candidate] (108.041 ms) : 0, 108041

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mattalp/baseline-flarereporter-smoketesting
git_commit_date	1756820968	1756826481
git_commit_sha	`890497c`	`2f8c79e`
release_version	1.53.0-SNAPSHOT~890497c9aa	1.53.0-SNAPSHOT~2f8c79eb55

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756828011	1756828011
ci_job_id	1109209809	1109209809
ci_pipeline_id	75310380	75310380
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-klo2jn80 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-klo2jn80 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 4 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load	better [-1229.077µs; -618.682µs] or [-8.375%; -4.216%]	unstable [-11.848op/s; +54.410op/s] or [-3.732%; +17.140%]	13.752ms	338.719op/s	14.675ms	317.438op/s
scenario:load:petclinic:profiling:high_load	worse [+2.466ms; +3.440ms] or [+5.301%; +7.396%]	unstable [-13.210op/s; +1.210op/s] or [-13.125%; +1.202%]	49.464ms	94.650op/s	46.511ms	100.650op/s
scenario:load:petclinic:no_agent:high_load	worse [+0.790ms; +1.433ms] or [+2.152%; +3.901%]	unstable [-12.717op/s; +5.292op/s] or [-9.989%; +4.157%]	37.832ms	123.600op/s	36.721ms	127.312op/s
scenario:load:petclinic:tracing:high_load	worse [+1.376ms; +2.207ms] or [+3.143%; +5.040%]	unstable [-10.089op/s; +4.339op/s] or [-9.441%; +4.060%]	45.577ms	103.987op/s	43.786ms	106.862op/s
scenario:load:petclinic:appsec:high_load	worse [+1.002ms; +1.905ms] or [+2.157%; +4.102%]	unstable [-10.358op/s; +4.233op/s] or [-10.276%; +4.200%]	47.895ms	97.737op/s	46.442ms	100.800op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~2f8c79eb55, baseline=1.53.0-SNAPSHOT~890497c9aa
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.31 ms) : 4252, 4369
.   : milestone, 4310,
iast (9.584 ms) : 9424, 9744
.   : milestone, 9584,
iast_FULL (14.675 ms) : 14382, 14969
.   : milestone, 14675,
iast_GLOBAL (10.272 ms) : 10080, 10464
.   : milestone, 10272,
profiling (8.719 ms) : 8581, 8857
.   : milestone, 8719,
tracing (7.422 ms) : 7319, 7526
.   : milestone, 7422,
section candidate
no_agent (4.238 ms) : 4192, 4285
.   : milestone, 4238,
iast (9.423 ms) : 9264, 9582
.   : milestone, 9423,
iast_FULL (13.752 ms) : 13479, 14024
.   : milestone, 13752,
iast_GLOBAL (10.386 ms) : 10201, 10571
.   : milestone, 10386,
profiling (8.732 ms) : 8589, 8875
.   : milestone, 8732,
tracing (7.471 ms) : 7363, 7580
.   : milestone, 7471,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.31 ms [4.252 ms, 4.369 ms]	-
iast	9.584 ms [9.424 ms, 9.744 ms]	5.274 ms (122.3%)
iast_FULL	14.675 ms [14.382 ms, 14.969 ms]	10.365 ms (240.5%)
iast_GLOBAL	10.272 ms [10.08 ms, 10.464 ms]	5.962 ms (138.3%)
profiling	8.719 ms [8.581 ms, 8.857 ms]	4.409 ms (102.3%)
tracing	7.422 ms [7.319 ms, 7.526 ms]	3.112 ms (72.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.238 ms [4.192 ms, 4.285 ms]	-
iast	9.423 ms [9.264 ms, 9.582 ms]	5.185 ms (122.3%)
iast_FULL	13.752 ms [13.479 ms, 14.024 ms]	9.513 ms (224.5%)
iast_GLOBAL	10.386 ms [10.201 ms, 10.571 ms]	6.148 ms (145.1%)
profiling	8.732 ms [8.589 ms, 8.875 ms]	4.494 ms (106.0%)
tracing	7.471 ms [7.363 ms, 7.58 ms]	3.233 ms (76.3%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~2f8c79eb55, baseline=1.53.0-SNAPSHOT~890497c9aa
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.721 ms) : 36429, 37012
.   : milestone, 36721,
appsec (46.442 ms) : 46022, 46862
.   : milestone, 46442,
code_origins (44.397 ms) : 44022, 44772
.   : milestone, 44397,
iast (44.383 ms) : 44004, 44763
.   : milestone, 44383,
profiling (46.511 ms) : 46055, 46967
.   : milestone, 46511,
tracing (43.786 ms) : 43420, 44152
.   : milestone, 43786,
section candidate
no_agent (37.832 ms) : 37527, 38138
.   : milestone, 37832,
appsec (47.895 ms) : 47476, 48315
.   : milestone, 47895,
code_origins (45.158 ms) : 44784, 45531
.   : milestone, 45158,
iast (45.356 ms) : 44954, 45759
.   : milestone, 45356,
profiling (49.464 ms) : 49014, 49913
.   : milestone, 49464,
tracing (45.577 ms) : 45173, 45982
.   : milestone, 45577,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.721 ms [36.429 ms, 37.012 ms]	-
appsec	46.442 ms [46.022 ms, 46.862 ms]	9.721 ms (26.5%)
code_origins	44.397 ms [44.022 ms, 44.772 ms]	7.676 ms (20.9%)
iast	44.383 ms [44.004 ms, 44.763 ms]	7.663 ms (20.9%)
profiling	46.511 ms [46.055 ms, 46.967 ms]	9.79 ms (26.7%)
tracing	43.786 ms [43.42 ms, 44.152 ms]	7.065 ms (19.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.832 ms [37.527 ms, 38.138 ms]	-
appsec	47.895 ms [47.476 ms, 48.315 ms]	10.063 ms (26.6%)
code_origins	45.158 ms [44.784 ms, 45.531 ms]	7.326 ms (19.4%)
iast	45.356 ms [44.954 ms, 45.759 ms]	7.524 ms (19.9%)
profiling	49.464 ms [49.014 ms, 49.913 ms]	11.632 ms (30.7%)
tracing	45.577 ms [45.173 ms, 45.982 ms]	7.745 ms (20.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mattalp/baseline-flarereporter-smoketesting
git_commit_date	1756820968	1756826481
git_commit_sha	`890497c`	`2f8c79e`
release_version	1.53.0-SNAPSHOT~890497c9aa	1.53.0-SNAPSHOT~2f8c79eb55

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1756828520	1756828520
ci_job_id	1109209811	1109209811
ci_pipeline_id	75310380	75310380
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-2-upskjs8r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-2-upskjs8r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~2f8c79eb55, baseline=1.53.0-SNAPSHOT~890497c9aa
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (3.667 ms) : 3449, 3886
.   : milestone, 3667,
iast (2.215 ms) : 2152, 2277
.   : milestone, 2215,
iast_GLOBAL (2.261 ms) : 2198, 2325
.   : milestone, 2261,
profiling (2.072 ms) : 2021, 2124
.   : milestone, 2072,
tracing (2.028 ms) : 1979, 2077
.   : milestone, 2028,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (3.671 ms) : 3453, 3889
.   : milestone, 3671,
iast (2.214 ms) : 2151, 2277
.   : milestone, 2214,
iast_GLOBAL (2.27 ms) : 2205, 2334
.   : milestone, 2270,
profiling (2.06 ms) : 2009, 2111
.   : milestone, 2060,
tracing (2.033 ms) : 1984, 2083
.   : milestone, 2033,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.477 ms [1.465 ms, 1.488 ms]	-
appsec	3.667 ms [3.449 ms, 3.886 ms]	2.191 ms (148.4%)
iast	2.215 ms [2.152 ms, 2.277 ms]	737.849 µs (50.0%)
iast_GLOBAL	2.261 ms [2.198 ms, 2.325 ms]	784.671 µs (53.1%)
profiling	2.072 ms [2.021 ms, 2.124 ms]	595.658 µs (40.3%)
tracing	2.028 ms [1.979 ms, 2.077 ms]	551.712 µs (37.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.464 ms, 1.487 ms]	-
appsec	3.671 ms [3.453 ms, 3.889 ms]	2.195 ms (148.8%)
iast	2.214 ms [2.151 ms, 2.277 ms]	738.756 µs (50.1%)
iast_GLOBAL	2.27 ms [2.205 ms, 2.334 ms]	794.148 µs (53.8%)
profiling	2.06 ms [2.009 ms, 2.111 ms]	584.597 µs (39.6%)
tracing	2.033 ms [1.984 ms, 2.083 ms]	557.709 µs (37.8%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~2f8c79eb55, baseline=1.53.0-SNAPSHOT~890497c9aa
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.557 s) : 15557000, 15557000
.   : milestone, 15557000,
appsec (15.018 s) : 15018000, 15018000
.   : milestone, 15018000,
iast (18.613 s) : 18613000, 18613000
.   : milestone, 18613000,
iast_GLOBAL (18.074 s) : 18074000, 18074000
.   : milestone, 18074000,
profiling (15.461 s) : 15461000, 15461000
.   : milestone, 15461000,
tracing (14.867 s) : 14867000, 14867000
.   : milestone, 14867000,
section candidate
no_agent (15.473 s) : 15473000, 15473000
.   : milestone, 15473000,
appsec (14.972 s) : 14972000, 14972000
.   : milestone, 14972000,
iast (18.109 s) : 18109000, 18109000
.   : milestone, 18109000,
iast_GLOBAL (18.266 s) : 18266000, 18266000
.   : milestone, 18266000,
profiling (15.975 s) : 15975000, 15975000
.   : milestone, 15975000,
tracing (14.795 s) : 14795000, 14795000
.   : milestone, 14795000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.557 s [15.557 s, 15.557 s]	-
appsec	15.018 s [15.018 s, 15.018 s]	-539.0 ms (-3.5%)
iast	18.613 s [18.613 s, 18.613 s]	3.056 s (19.6%)
iast_GLOBAL	18.074 s [18.074 s, 18.074 s]	2.517 s (16.2%)
profiling	15.461 s [15.461 s, 15.461 s]	-96.0 ms (-0.6%)
tracing	14.867 s [14.867 s, 14.867 s]	-690.0 ms (-4.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.473 s [15.473 s, 15.473 s]	-
appsec	14.972 s [14.972 s, 14.972 s]	-501.0 ms (-3.2%)
iast	18.109 s [18.109 s, 18.109 s]	2.636 s (17.0%)
iast_GLOBAL	18.266 s [18.266 s, 18.266 s]	2.793 s (18.1%)
profiling	15.975 s [15.975 s, 15.975 s]	502.0 ms (3.2%)
tracing	14.795 s [14.795 s, 14.795 s]	-678.0 ms (-4.4%)

Squashed 9 commits

dd-smoke-tests/tracer-flare/src/test/groovy/datadog/smoketest/TracerFlareSmokeTest.groovy

jbachorik

Please, make the wait for flare more robust

jbachorik

Thanks!

MattAlp requested a review from a team as a code owner August 29, 2025 18:36

MattAlp added the type: enhancement Enhancements and improvements label Aug 29, 2025

MattAlp requested a review from a team as a code owner August 29, 2025 18:36

MattAlp added the comp: testing Testing label Aug 29, 2025

MattAlp requested review from amarziali, PerfectSlayer and AlexeyKuznetsov-DD August 29, 2025 18:36

MattAlp changed the title ~~Create a flare reporter smoke test~~ Create a flare contents smoke test Aug 29, 2025

MattAlp requested a review from jbachorik August 29, 2025 19:20

AlexeyKuznetsov-DD approved these changes Aug 29, 2025

View reviewed changes

MattAlp force-pushed the mattalp/baseline-flarereporter-smoketesting branch from adb24a8 to 55b00bf Compare August 29, 2025 20:10

Initial smoke tests for tracer flare functionality

5f837f1

Squashed 9 commits

MattAlp force-pushed the mattalp/baseline-flarereporter-smoketesting branch from 90f78ef to 5f837f1 Compare August 29, 2025 22:48

jbachorik reviewed Sep 1, 2025

View reviewed changes

dd-smoke-tests/tracer-flare/src/test/groovy/datadog/smoketest/TracerFlareSmokeTest.groovy Outdated Show resolved Hide resolved

jbachorik requested changes Sep 1, 2025

View reviewed changes

Replace naive wait with file watching

2f8c79e

MattAlp requested a review from jbachorik September 2, 2025 15:27

jbachorik approved these changes Sep 2, 2025

View reviewed changes

MattAlp merged commit 0a4e9d0 into master Sep 2, 2025
503 checks passed

MattAlp deleted the mattalp/baseline-flarereporter-smoketesting branch September 2, 2025 16:13

github-actions bot added this to the 1.53.0 milestone Sep 2, 2025

MattAlp mentioned this pull request Sep 2, 2025

Increase flare reporter smoke test duration #9452

Merged

mcculls added the tag: no release notes Changes to exclude from release notes label Sep 4, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Create a flare contents smoke test #9440

Create a flare contents smoke test #9440

Uh oh!

MattAlp commented Aug 29, 2025 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 bot commented Aug 29, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 29, 2025 •

edited

Loading

Uh oh!

Uh oh!

jbachorik left a comment

Uh oh!

jbachorik left a comment

Uh oh!

Uh oh!

Uh oh!

Create a flare contents smoke test #9440

Create a flare contents smoke test #9440

Uh oh!

Conversation

MattAlp commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-datadog-prod-us1 bot commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

jbachorik left a comment

Choose a reason for hiding this comment

Uh oh!

jbachorik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MattAlp commented Aug 29, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Aug 29, 2025 •

edited

Loading

pr-commenter bot commented Aug 29, 2025 •

edited

Loading