feat(civisibility): add test.final_status tag for adjusted test outcomes

[tonyredondo]

What does this PR do?

JIRA: SDTEST-2986

Adds a new test.final_status tag to CI Visibility test spans that indicates the final adjusted status for a test after considering all executions (including retries). This helps distinguish between the raw test outcome and the final computed result.

The priority order for determining final status is:

1. Quarantined/Disabled → skip (management directives take precedence)
2. Any pass in retries → pass (a single pass means the test ultimately passed)
3. Any fail (no pass) → fail (all executions failed)
4. Current is skip → skip (single-execution skip)
5. Default → fail

Key Changes

• New constant: TestFinalStatus = "test.final_status" in constants/test_tags.go
• New file: final_status.go with helper functions:
  • calculateFinalStatus() - computes the final status based on execution outcomes
  • isFinalExecution() - determines if current execution is the last one (no more retries)
  • computeAdjustedRetryCount() - mirrors retry count logic for prediction
  • willRetryAfterExecution() - predicts if another retry will happen
• Metadata tracking: Added anyExecutionPassed, anyExecutionFailed, remainingRetries, isEfdInParallel, and skipReason fields to testExecutionMetadata
• Flag propagation: isEfdInParallel and hasAdditionalFeatureWrapper are propagated to subtests
• Skip reason preservation: Store skip reason when instrumentCloseAndSkip returns early, use it in defer block
• Parallel EFD handling: Option A - don't set test.final_status for parallel EFD (impossible to determine final execution)
• ATF precedence: ATF tests compute test.final_status even when parallel EFD is enabled
• Test coverage: Added assertions in testcontroller_test.go and subtestcontroller_test.go

Motivation

The test.final_status tag allows CI Visibility consumers to understand the ultimate outcome of a test after all retries and management directives have been applied. This is particularly useful for:

• Flaky test detection (distinguishing flaky passes from solid passes)
• Test management reporting (showing adjusted status for quarantined/disabled tests)
• Retry analysis (understanding if retries helped the test pass)

Test Plan

• All existing test scenarios pass (7 scenarios in testcontroller_test.go)
• All subtest scenarios pass (10 scenarios in subtestcontroller_test.go)
• Added test.final_status assertions for:
  • Pass case (single execution and after retries)
  • Skip case (single execution, disabled, quarantined)
  • Parallel EFD (Option A - no final_status set)
  • ITR-skipped tests
  • Test management tests (disabled/quarantined with ATF)
  • Subtests (baseline, disabled, quarantined, parent quarantined)

Reviewer's Checklist

• Changed code has unit tests for its functionality at or near 100% coverage.
• System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
• There is a benchmark for any new code, or changes to existing code.
• If this interacts with the agent in a new way, a system test has been added.
• New code is free of linting errors. You can check this by running ./scripts/lint.sh locally.
• Add an appropriate team label so this PR gets put in the right place for the release notes.
• Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

[codecov]

Codecov Report

❌ Patch coverage is 72.43590% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.15%. Comparing base (fa72b69) to head (e2addd8).

Files with missing lines	Patch %	Lines
...egrations/gotesting/instrumentation_orchestrion.go	33.33%	27 Missing and 3 partials ⚠️
...ivisibility/integrations/gotesting/final_status.go	81.66%	8 Missing and 3 partials ⚠️
...nal/civisibility/integrations/gotesting/testing.go	94.73%	0 Missing and 2 partials ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
...sibility/integrations/gotesting/instrumentation.go	82.43% <100.00%> (+0.55%)	⬆️
...nal/civisibility/integrations/gotesting/testing.go	59.38% <94.73%> (+2.86%)	⬆️
...ivisibility/integrations/gotesting/final_status.go	81.66% <81.66%> (ø)
...egrations/gotesting/instrumentation_orchestrion.go	29.93% <33.33%> (+0.66%)	⬆️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-01-27 15:02:19

Comparing candidate commit e2addd8 in PR branch tony/civiz-test-final-status with baseline commit fa72b69 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 157 metrics, 7 unstable metrics.

[anmarchenko]

this would not be right for the attempt to fix flow:

If the test is in attempt-to-fix flow, it should be reported to the test framework as failed if any of the retries failed - so flaky test that is still flaky in attempt-to-fix fails the build.

I was recently fixing this issue for the Ruby library.

[tonyredondo]

oh, great, changed in e2addd8