fix(test-optimization): jest instrumentation applies auto test retries on top of other retry mechanisms

[calvinbayer]

What does this PR do?

Introduce an order of priority for applying retry mechanisms:

• Attempt to Fix
• Early Flake Detection
• Auto Test Retries

This change aligns the behavior of the Jest instrumentation with how other tracers do it. It also decreases the number of retries and thus the performance overhead introduced by instrumenting tests with Test Optimization. E.g., before this change, if both EFD and ATR were enabled, a new failing test would be executed 66 times: (1 execution + 10 EFD retries) * (1 execution + 5 ATR retries). After this change, ATR retries will not be applied anymore and this test is only executed 11 times as expected.

Motivation

Working on adding the @test.final_status tag had unexpected side effects with multiple retry mechanisms being applied.

[github-actions]

Overall package size

Self size: 4.39 MB
Deduped: 5.21 MB
No deduping: 5.21 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 2.0.0 | 68.46 kB | 797.03 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.87%. Comparing base (e48b161) to head (13aae60).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7251      +/-   ##
==========================================
- Coverage   84.87%   84.87%   -0.01%     
==========================================
  Files         532      531       -1     
  Lines       22654    22651       -3     
==========================================
- Hits        19227    19224       -3     
  Misses       3427     3427              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[datadog-official]

⚠️ Tests

✨ Fix all issues with Cursor

⚠️ Warnings

🧪 2 Tests failed

IAST - overhead-controller - integration "after each" hook for "should report vulnerability only in the first request" from vulnerability sampling algorithm (Datadog) (Fix with Cursor)

Cannot read properties of undefined (reading 'kill')

TypeError: Cannot read properties of undefined (reading 'kill')
    at Context.<anonymous> (packages/dd-trace/test/appsec/iast/overhead-controller.integration.spec.js:27:10)
    at process.processImmediate (node:internal/timers:504:21)

IAST - overhead-controller - integration vulnerability sampling algorithm "before each" hook for "should report vulnerability only in the first request" from vulnerability sampling algorithm (Datadog) (Fix with Cursor)

Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/runner/work/dd-trace-js/dd-trace-js/packages/dd-trace/test/appsec/iast/overhead-controller.integration.spec.js)

Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/runner/work/dd-trace-js/dd-trace-js/packages/dd-trace/test/appsec/iast/overhead-controller.integration.spec.js)
    at listOnTimeout (node:internal/timers:605:17)
    at process.processTimers (node:internal/timers:541:7)

ℹ️ Info

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 13aae60 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-01-15 10:13:41

Comparing candidate commit 13aae60 in PR branch calvin.bayer/fix/retry-mechanism-priority-order with baseline commit e48b161 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 288 metrics, 32 unstable metrics.

[juan-fernandez]

👏 nice! Only minor comments, otherwise LGTM

[juan-fernandez]

could we try to make the test async instead of using done? I've been trying to get rid of done

[juan-fernandez]

nitpick

Suggested change

	}
	)
	})

[juan-fernandez]

should we also check that attempt to fix takes precedence over retries triggered by impacted_tests_enabled: true?

[juan-fernandez]

nitpick: this is redundant as it's already a requirement for isAttemptToFix

Suggested change

this.isTestManagementTestsEnabled &&