[calvinbayer]

What Does This Do

Report test.test_management.no_auto_retry and test.test_management.prevent_failure_suppression for native Weaver TestName.tagged declarations using the public TestAutomationExclusions constants. Capture a fresh string-only plan snapshot for each suite execution and clean it up when the suite finishes or reporter stops.

Motivation

Keep exclusion declaration metadata visible for Weaver, including reported native failure and skip outcomes. Weaver currently supports none of Datadog ATR, EFD, Attempt to Fix, quarantine, or disabling; this change does not enable unsupported automation capabilities.

Additional Notes

Stacked on codex/munit-automation-exclusions. Validation: module spotlessApply and instrumented WeaverTest.test automation exclusions* passed, proving both true-only annotation tags on failure and developer-skip spans with unchanged native status. No refactor commit was needed.

Snapshot dependency for inherited physical source paths: DataDog/test-environment#604.

test-environment-branch: codex/java-inherited-test-source-snapshots

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	57.25	56.55	$\color{red}{\blacktriangle}$ +0.70	56.55	$\color{red}{\blacktriangle}$ +0.70	185/427
agentEvpProxy	57.52	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	36.06	37.15	$\color{green}{\blacktriangledown}$ -1.09	37.15	$\color{green}{\blacktriangledown}$ -1.09	92/225
agentless	37.60	37.15	$\color{red}{\blacktriangle}$ +0.45	37.15	$\color{red}{\blacktriangle}$ +0.45	92/198
agentlessCodeCoverage	45.94	46.30	$\color{green}{\blacktriangledown}$ -0.36	45.38	$\color{red}{\blacktriangle}$ +0.56	92/197
agentlessLineCoverage	56.49	56.55	$\color{green}{\blacktriangledown}$ -0.06	56.55	$\color{green}{\blacktriangledown}$ -0.06	92/197

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[datadog-prod-us1-3]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.94% (+0.00%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 948fa24 | Docs | Give us feedback!}

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	88.78	89.58	$\color{green}{\blacktriangledown}$ -0.80	89.58	$\color{green}{\blacktriangledown}$ -0.80	92/226
agentless	82.48	82.69	$\color{green}{\blacktriangledown}$ -0.21	82.69	$\color{green}{\blacktriangledown}$ -0.21	92/197
agentlessCodeCoverage	96.42	97.04	$\color{green}{\blacktriangledown}$ -0.62	97.04	$\color{green}{\blacktriangledown}$ -0.62	91/196
agentlessLineCoverage	112.27	113.88	$\color{green}{\blacktriangledown}$ -1.61	113.88	$\color{green}{\blacktriangledown}$ -1.61	91/196

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	14.54	10.54	$\color{red}{\blacktriangle}$ +4.00	10.33	$\color{red}{\blacktriangle}$ +4.21	93/225
agentless	10.70	9.92	$\color{red}{\blacktriangle}$ +0.78	9.92	$\color{red}{\blacktriangle}$ +0.78	92/195
agentlessCodeCoverage	17.29	17.38	$\color{green}{\blacktriangledown}$ -0.09	17.73	$\color{green}{\blacktriangledown}$ -0.44	84/180

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	21.90	22.54	$\color{green}{\blacktriangledown}$ -0.64	22.54	$\color{green}{\blacktriangledown}$ -0.64	122/257
agentless	18.79	19.59	$\color{green}{\blacktriangledown}$ -0.80	19.59	$\color{green}{\blacktriangledown}$ -0.80	117/220
agentlessCodeCoverage	19.75	20.39	$\color{green}{\blacktriangledown}$ -0.64	20.39	$\color{green}{\blacktriangledown}$ -0.64	118/221
agentlessLineCoverage	26.08	27.52	$\color{green}{\blacktriangledown}$ -1.44	27.52	$\color{green}{\blacktriangledown}$ -1.44	81/156

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	10.07	9.73	$\color{red}{\blacktriangle}$ +0.34	9.92	$\color{red}{\blacktriangle}$ +0.15	89/194

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	12.71	13.67	$\color{green}{\blacktriangledown}$ -0.96	13.40	$\color{green}{\blacktriangledown}$ -0.69	104/239
agentless	11.63	12.37	$\color{green}{\blacktriangledown}$ -0.74	12.37	$\color{green}{\blacktriangledown}$ -0.74	104/209
agentlessCodeCoverage	14.86	15.72	$\color{green}{\blacktriangledown}$ -0.86	15.72	$\color{green}{\blacktriangledown}$ -0.86	104/209
agentlessLineCoverage	17.93	17.38	$\color{red}{\blacktriangle}$ +0.55	17.73	$\color{red}{\blacktriangle}$ +0.20	104/209

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	95.33	97.04	$\color{green}{\blacktriangledown}$ -1.71	95.12	$\color{red}{\blacktriangle}$ +0.21	107/241
agentless	91.34	91.39	$\color{green}{\blacktriangledown}$ -0.05	91.39	$\color{green}{\blacktriangledown}$ -0.05	107/213
agentlessCodeCoverage	101.33	101.00	$\color{red}{\blacktriangle}$ +0.33	99.00	$\color{red}{\blacktriangle}$ +2.33	107/213
agentlessLineCoverage	102.16	103.04	$\color{green}{\blacktriangledown}$ -0.88	101.00	$\color{red}{\blacktriangle}$ +1.16	119/231

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - okhttp

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	19.34	18.82	$\color{red}{\blacktriangle}$ +0.52	19.59	$\color{green}{\blacktriangledown}$ -0.25	107/240
agentless	18.06	18.45	$\color{green}{\blacktriangledown}$ -0.39	19.20	$\color{green}{\blacktriangledown}$ -1.14	106/215
agentlessCodeCoverage	23.35	21.65	$\color{red}{\blacktriangle}$ +1.70	22.54	$\color{red}{\blacktriangle}$ +0.81	108/219
agentlessLineCoverage	36.00	37.91	$\color{green}{\blacktriangledown}$ -1.91	38.67	$\color{green}{\blacktriangledown}$ -2.67	105/213

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - spring_boot

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	9.87	16.36	$\color{green}{\blacktriangledown}$ -6.49	16.36	$\color{green}{\blacktriangledown}$ -6.49	104/237
agentless	10.65	9.92	$\color{red}{\blacktriangle}$ +0.73	9.92	$\color{red}{\blacktriangle}$ +0.73	104/209
agentlessCodeCoverage	13.73	13.94	$\color{green}{\blacktriangledown}$ -0.21	13.67	$\color{red}{\blacktriangle}$ +0.06	104/211
agentlessLineCoverage	22.58	22.99	$\color{green}{\blacktriangledown}$ -0.41	22.54	$\color{red}{\blacktriangle}$ +0.04	104/209

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.76 s	14.66 s	[-0.1%; +1.4%] (no difference)
startup:insecure-bank:tracing:Agent	13.59 s	13.64 s	[-1.3%; +0.7%] (no difference)
startup:petclinic:appsec:Agent	17.72 s	17.52 s	[+0.2%; +2.1%] (maybe worse)
startup:petclinic:iast:Agent	17.50 s	17.59 s	[-1.3%; +0.2%] (no difference)
startup:petclinic:profiling:Agent	17.22 s	17.56 s	[-3.0%; -0.9%] (maybe better)
startup:petclinic:sca:Agent	17.55 s	17.62 s	[-1.4%; +0.5%] (no difference)
startup:petclinic:tracing:Agent	16.17 s	16.58 s	[-6.7%; +1.8%] (no difference)

Commit: 948fa243 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-19T22:22:30.987753Z	8091184	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 809118440e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-java

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	36.03	14.51	$\color{red}{\blacktriangle}$ +21.52	14.23	$\color{red}{\blacktriangle}$ +21.80	104/237
agentless	9.55	12.37	$\color{green}{\blacktriangledown}$ -2.82	12.87	$\color{green}{\blacktriangledown}$ -3.32	104/211
agentlessCodeCoverage	77.30	84.36	$\color{green}{\blacktriangledown}$ -7.06	81.05	$\color{green}{\blacktriangledown}$ -3.75	104/211
agentlessLineCoverage	125.12	120.92	$\color{red}{\blacktriangle}$ +4.20	123.36	$\color{red}{\blacktriangle}$ +1.76	104/212

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[datadog-prod-us1-3]

Weaver suites with customized names lose their declared exclusion metadata because snapshot capture and reporting use different suite identities. The mismatch also prevents snapshot cleanup at suite completion.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 8091184}

[datadog-prod-us1-3]

The shared object-factory retry test fails on TestNG 6.4 and 7.0.0 because it loads an unavailable interface before checking compatibility. Both configurations need to skip this unsupported case.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit df5a259}

[calvinbayer]

@autotest Review

[datadog-prod-us1-3]

More details

Exclusion metadata uses Weaver’s task identity, including provider-backed suites, while preserving native failure and skip outcomes. The static assessment supports reporting these declarations without enabling unsupported automation capabilities.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 0ba545b}

[calvinbayer]

@autotest Review

[datadog-prod-us1-3]

The new callback hook may omit exclusion metadata for Scala 2 Weaver cross-builds because it assumes an instance-method layout. The module’s Scala 3-only coverage does not resolve that compatibility concern.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 1c90131}

[calvinbayer]

@autotest Review

[datadog-prod-us1-3]

More details

Declaration-tag capture preserves Weaver’s native outcome handling. Source and bytecode inspection supported callback compatibility across the checked Weaver and Scala versions, with no actionable defects identified.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit eefd933}

[calvinbayer]

@autotest Review

[datadog-prod-us1-3]

More details

Source and dependency-bytecode inspection support exclusion-tag capture, lookup, and cleanup across the inspected Scala variants, including Weaver 0.8.4 and 0.13.0.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 948fa24}