Skip to content

Update impacted tests logic #8923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 28 commits into
base: master
Choose a base branch
from
Open

Update impacted tests logic #8923

wants to merge 28 commits into from

Conversation

daniel-mohedano
Copy link
Contributor

What Does This Do

Updates the logic behind the impacted tests feature:

  • Removes the backend request for changed files
  • Adds default_branch to the expected response from the Settings endpoint
  • Implements a best effort algorithm for calculating the base commit SHA (commit where the feature/source branch originally diverted from the base branch)

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@daniel-mohedano
remove backend request for diff
6980d02
@daniel-mohedano
update PRInfo building to use partial information from CI vendors
a7f738d
@daniel-mohedano
synchronize repo unshallow with future, avoids performing several times
a1238c0
@daniel-mohedano
add default_branch to settings response
cbe2a95
@daniel-mohedano
update settings api test
b861e7f
@daniel-mohedano
change repo unshallow synchronization from future-based to synchronized
cab317e
@daniel-mohedano
include pr info checks to CITagsProviderTest
e401549
@daniel-mohedano
add tests for prinfo merging
63e7236
@daniel-mohedano
Revert "update PRInfo building to use partial information from CI ven…
a109cfd 
…dors"

This reverts commit a7f738d.
@daniel-mohedano
Revert "include pr info checks to CITagsProviderTest"
4d18887 
This reverts commit e401549.
@daniel-mohedano
Revert "add tests for prinfo merging"
84e1c0f 
This reverts commit 63e7236.
@daniel-mohedano
base commit sha algorithm implementation
ec152b6
@daniel-mohedano
example git repo for base commit sha testing
1e843af
@daniel-mohedano
simplify algo implementation for easier read
fc82fcc
@daniel-mohedano
implement git diff based on best candidate for base commit sha
67339e9
@daniel-mohedano
add github actions cloning test
bfd3ad4
@daniel-mohedano
add several testing cases and cleaned up debug messages
5ddb58c
@daniel-mohedano
remove repo_origin folder
c558a86
@daniel-mohedano daniel-mohedano added type: enhancement comp: ci visibility Continuous Integration Visibility labels Jun 4, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jun 4, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/impacted-tests-update
git_commit_date 1749044153 1749212549
git_commit_sha 7787af7 99980be
release_version 1.50.0-SNAPSHOT~7787af738f 1.50.0-SNAPSHOT~99980be4b4
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1749214914 1749214914
ci_job_id 971762838 971762838
ci_pipeline_id 67170471 67170471
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-pz8nwqvr-project-304-concurrent-2-rxvhsx4k 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-pz8nwqvr-project-304-concurrent-2-rxvhsx4k 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 13 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~99980be4b4, baseline=1.50.0-SNAPSHOT~7787af738f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.024 s) : 0, 1023531
Total [baseline] (8.54 s) : 0, 8540195
Agent [candidate] (1.025 s) : 0, 1025145
Total [candidate] (8.537 s) : 0, 8536504
section iast
Agent [baseline] (1.15 s) : 0, 1150020
Total [baseline] (9.171 s) : 0, 9170614
Agent [candidate] (1.156 s) : 0, 1156267
Total [candidate] (9.194 s) : 0, 9193602
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.152 s) : 0, 1151749
Total [baseline] (9.162 s) : 0, 9162217
Agent [candidate] (1.16 s) : 0, 1160167
Total [candidate] (9.164 s) : 0, 9163731
section iast_TELEMETRY_OFF
Agent [baseline] (1.15 s) : 0, 1150461
Total [baseline] (9.273 s) : 0, 9272691
Agent [candidate] (1.149 s) : 0, 1148715
Total [candidate] (9.238 s) : 0, 9237718
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.024 s -
Agent iast 1.15 s 126.489 ms (12.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.152 s 128.217 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.15 s 126.929 ms (12.4%)
Total tracing 8.54 s -
Total iast 9.171 s 630.42 ms (7.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.162 s 622.023 ms (7.3%)
Total iast_TELEMETRY_OFF 9.273 s 732.497 ms (8.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.025 s -
Agent iast 1.156 s 131.122 ms (12.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.16 s 135.022 ms (13.2%)
Agent iast_TELEMETRY_OFF 1.149 s 123.57 ms (12.1%)
Total tracing 8.537 s -
Total iast 9.194 s 657.098 ms (7.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.164 s 627.228 ms (7.3%)
Total iast_TELEMETRY_OFF 9.238 s 701.214 ms (8.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~99980be4b4, baseline=1.50.0-SNAPSHOT~7787af738f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.408 ms) : 0, 684408
BytebuddyAgent [candidate] (684.506 ms) : 0, 684506
GlobalTracer [baseline] (240.006 ms) : 0, 240006
GlobalTracer [candidate] (240.771 ms) : 0, 240771
AppSec [baseline] (59.02 ms) : 0, 59020
AppSec [candidate] (57.899 ms) : 0, 57899
Debugger [baseline] (6.099 ms) : 0, 6099
Debugger [candidate] (6.2 ms) : 0, 6200
Remote Config [baseline] (729.567 µs) : 0, 730
Remote Config [candidate] (738.855 µs) : 0, 739
Telemetry [baseline] (9.665 ms) : 0, 9665
Telemetry [candidate] (11.404 ms) : 0, 11404
section iast
BytebuddyAgent [baseline] (802.269 ms) : 0, 802269
BytebuddyAgent [candidate] (807.526 ms) : 0, 807526
GlobalTracer [baseline] (230.298 ms) : 0, 230298
GlobalTracer [candidate] (231.143 ms) : 0, 231143
AppSec [baseline] (53.34 ms) : 0, 53340
AppSec [candidate] (54.807 ms) : 0, 54807
Debugger [baseline] (5.988 ms) : 0, 5988
Debugger [candidate] (5.966 ms) : 0, 5966
Remote Config [baseline] (594.125 µs) : 0, 594
Remote Config [candidate] (602.487 µs) : 0, 602
Telemetry [baseline] (7.858 ms) : 0, 7858
Telemetry [candidate] (7.918 ms) : 0, 7918
IAST [baseline] (26.139 ms) : 0, 26139
IAST [candidate] (24.609 ms) : 0, 24609
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (802.845 ms) : 0, 802845
BytebuddyAgent [candidate] (809.805 ms) : 0, 809805
GlobalTracer [baseline] (230.705 ms) : 0, 230705
GlobalTracer [candidate] (231.936 ms) : 0, 231936
AppSec [baseline] (54.412 ms) : 0, 54412
AppSec [candidate] (50.52 ms) : 0, 50520
Debugger [baseline] (5.975 ms) : 0, 5975
Debugger [candidate] (5.957 ms) : 0, 5957
Remote Config [baseline] (603.867 µs) : 0, 604
Remote Config [candidate] (596.225 µs) : 0, 596
Telemetry [baseline] (7.994 ms) : 0, 7994
Telemetry [candidate] (8.061 ms) : 0, 8061
IAST [baseline] (25.623 ms) : 0, 25623
IAST [candidate] (29.597 ms) : 0, 29597
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (800.68 ms) : 0, 800680
BytebuddyAgent [candidate] (799.595 ms) : 0, 799595
GlobalTracer [baseline] (231.901 ms) : 0, 231901
GlobalTracer [candidate] (231.184 ms) : 0, 231184
AppSec [baseline] (51.285 ms) : 0, 51285
AppSec [candidate] (51.5 ms) : 0, 51500
Debugger [baseline] (6.009 ms) : 0, 6009
Debugger [candidate] (6.041 ms) : 0, 6041
Remote Config [baseline] (612.944 µs) : 0, 613
Remote Config [candidate] (608.83 µs) : 0, 609
Telemetry [baseline] (7.822 ms) : 0, 7822
Telemetry [candidate] (7.956 ms) : 0, 7956
IAST [baseline] (28.55 ms) : 0, 28550
IAST [candidate] (28.286 ms) : 0, 28286
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~99980be4b4, baseline=1.50.0-SNAPSHOT~7787af738f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.031 s) : 0, 1031354
Total [baseline] (11.25 s) : 0, 11249642
Agent [candidate] (1.024 s) : 0, 1024299
Total [candidate] (11.087 s) : 0, 11087002
section appsec
Agent [baseline] (1.169 s) : 0, 1168986
Total [baseline] (11.256 s) : 0, 11255585
Agent [candidate] (1.163 s) : 0, 1163440
Total [candidate] (11.256 s) : 0, 11255709
section iast
Agent [baseline] (1.15 s) : 0, 1150379
Total [baseline] (11.398 s) : 0, 11397876
Agent [candidate] (1.152 s) : 0, 1151667
Total [candidate] (11.341 s) : 0, 11340894
section profiling
Agent [baseline] (1.265 s) : 0, 1265147
Total [baseline] (11.482 s) : 0, 11482339
Agent [candidate] (1.275 s) : 0, 1275269
Total [candidate] (11.484 s) : 0, 11484172
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.031 s -
Agent appsec 1.169 s 137.632 ms (13.3%)
Agent iast 1.15 s 119.025 ms (11.5%)
Agent profiling 1.265 s 233.793 ms (22.7%)
Total tracing 11.25 s -
Total appsec 11.256 s 5.943 ms (0.1%)
Total iast 11.398 s 148.233 ms (1.3%)
Total profiling 11.482 s 232.697 ms (2.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.024 s -
Agent appsec 1.163 s 139.141 ms (13.6%)
Agent iast 1.152 s 127.368 ms (12.4%)
Agent profiling 1.275 s 250.97 ms (24.5%)
Total tracing 11.087 s -
Total appsec 11.256 s 168.707 ms (1.5%)
Total iast 11.341 s 253.892 ms (2.3%)
Total profiling 11.484 s 397.17 ms (3.6%) 
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~99980be4b4, baseline=1.50.0-SNAPSHOT~7787af738f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.213 ms) : 0, 688213
BytebuddyAgent [candidate] (684.277 ms) : 0, 684277
GlobalTracer [baseline] (241.581 ms) : 0, 241581
GlobalTracer [candidate] (240.817 ms) : 0, 240817
AppSec [baseline] (57.259 ms) : 0, 57259
AppSec [candidate] (57.275 ms) : 0, 57275
Debugger [baseline] (6.93 ms) : 0, 6930
Debugger [candidate] (6.171 ms) : 0, 6171
Remote Config [baseline] (761.223 µs) : 0, 761
Remote Config [candidate] (733.625 µs) : 0, 734
Telemetry [baseline] (12.822 ms) : 0, 12822
Telemetry [candidate] (11.386 ms) : 0, 11386
section appsec
BytebuddyAgent [baseline] (704.416 ms) : 0, 704416
BytebuddyAgent [candidate] (700.394 ms) : 0, 700394
GlobalTracer [baseline] (238.813 ms) : 0, 238813
GlobalTracer [candidate] (238.01 ms) : 0, 238010
AppSec [baseline] (176.963 ms) : 0, 176963
AppSec [candidate] (176.558 ms) : 0, 176558
Debugger [baseline] (5.968 ms) : 0, 5968
Debugger [candidate] (5.982 ms) : 0, 5982
Remote Config [baseline] (640.277 µs) : 0, 640
Remote Config [candidate] (618.597 µs) : 0, 619
Telemetry [baseline] (7.383 ms) : 0, 7383
Telemetry [candidate] (7.32 ms) : 0, 7320
IAST [baseline] (22.173 ms) : 0, 22173
IAST [candidate] (21.847 ms) : 0, 21847
section iast
BytebuddyAgent [baseline] (801.982 ms) : 0, 801982
BytebuddyAgent [candidate] (802.9 ms) : 0, 802900
GlobalTracer [baseline] (230.551 ms) : 0, 230551
GlobalTracer [candidate] (231.011 ms) : 0, 231011
AppSec [baseline] (52.695 ms) : 0, 52695
AppSec [candidate] (52.851 ms) : 0, 52851
Debugger [baseline] (5.968 ms) : 0, 5968
Debugger [candidate] (5.971 ms) : 0, 5971
Remote Config [baseline] (601.974 µs) : 0, 602
Remote Config [candidate] (601.903 µs) : 0, 602
Telemetry [baseline] (7.951 ms) : 0, 7951
Telemetry [candidate] (7.893 ms) : 0, 7893
IAST [baseline] (27.043 ms) : 0, 27043
IAST [candidate] (26.87 ms) : 0, 26870
section profiling
BytebuddyAgent [baseline] (674.762 ms) : 0, 674762
BytebuddyAgent [candidate] (680.444 ms) : 0, 680444
GlobalTracer [baseline] (360.699 ms) : 0, 360699
GlobalTracer [candidate] (362.47 ms) : 0, 362470
AppSec [baseline] (61.53 ms) : 0, 61530
AppSec [candidate] (62.547 ms) : 0, 62547
Debugger [baseline] (6.066 ms) : 0, 6066
Debugger [candidate] (6.136 ms) : 0, 6136
Remote Config [baseline] (654.217 µs) : 0, 654
Remote Config [candidate] (637.527 µs) : 0, 638
Telemetry [baseline] (8.113 ms) : 0, 8113
Telemetry [candidate] (8.131 ms) : 0, 8131
ProfilingAgent [baseline] (102.531 ms) : 0, 102531
ProfilingAgent [candidate] (103.71 ms) : 0, 103710
Profiling [baseline] (102.555 ms) : 0, 102555
Profiling [candidate] (103.735 ms) : 0, 103735
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-06-06T12:42:47 2025-06-06T12:50:16
git_branch master daniel.mohedano/impacted-tests-update
git_commit_date 1749044153 1749212549
git_commit_sha 7787af7 99980be
release_version 1.50.0-SNAPSHOT~7787af738f 1.50.0-SNAPSHOT~99980be4b4
start_time 2025-06-06T12:40:45 2025-06-06T12:49:29
See matching parameters
Baseline Candidate
application petclinic petclinic
ci_job_date 1749214216 1749214216
ci_job_id 971762839 971762839
ci_pipeline_id 67170471 67170471
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-66eemyw1-project-304-concurrent-1-a78sppwg 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-66eemyw1-project-304-concurrent-1-a78sppwg 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
thresholds_or_results results results
variant appsec appsec

Summary

Found 0 performance improvements and 6 performance regressions! Performance is the same for 0 metrics, 8 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:appsec unstable
[+76.978ms; +80.072ms] or [+1017.140%; +1058.018%]		 worse
[-609.278op/s; -579.577op/s] or [-93.676%; -89.110%]		 86.093ms 55.979op/s 7.568ms 650.407op/s
scenario:load:petclinic:appsec_no_iast unstable
[+82.025ms; +85.794ms] or [+1011.645%; +1058.130%]		 worse
[-569.419op/s; -548.066op/s] or [-93.242%; -89.746%]		 92.017ms 51.944op/s 8.108ms 610.687op/s
scenario:load:petclinic:iast unstable
[+86.967ms; +92.273ms] or [+812.214%; +861.768%]		 worse
[-431.713op/s; -394.903op/s] or [-93.678%; -85.690%]		 100.327ms 47.542op/s 10.707ms 460.850op/s
scenario:load:petclinic:no_agent unstable
[+50.688ms; +52.467ms] or [+644.944%; +667.586%]		 worse
[-559.914op/s; -525.111op/s] or [-89.586%; -84.018%]		 59.437ms 82.487op/s 7.859ms 625.000op/s
scenario:load:petclinic:profiling unstable
[+66.648ms; +69.122ms] or [+805.149%; +835.034%]		 worse
[-545.805op/s; -520.158op/s] or [-91.422%; -87.127%]		 76.162ms 64.033op/s 8.278ms 597.015op/s
scenario:load:petclinic:tracing unstable
[+57.569ms; +59.605ms] or [+724.580%; +750.202%]		 worse
[-560.806op/s; -533.004op/s] or [-90.430%; -85.947%]		 66.532ms 73.250op/s 7.945ms 620.155op/s

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/impacted-tests-update
git_commit_date 1749044153 1749212549
git_commit_sha 7787af7 99980be
release_version 1.50.0-SNAPSHOT~7787af738f 1.50.0-SNAPSHOT~99980be4b4
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1749214443 1749214443
ci_job_id 971762840 971762840
ci_pipeline_id 67170471 67170471
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-i2yz3wjx-project-304-concurrent-2-7myheysc 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-i2yz3wjx-project-304-concurrent-2-7myheysc 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~99980be4b4, baseline=1.50.0-SNAPSHOT~7787af738f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (2.389 ms) : 2341, 2438
.   : milestone, 2389,
iast (2.18 ms) : 2119, 2241
.   : milestone, 2180,
iast_GLOBAL (2.229 ms) : 2168, 2291
.   : milestone, 2229,
profiling (2.022 ms) : 1973, 2071
.   : milestone, 2022,
tracing (1.995 ms) : 1947, 2042
.   : milestone, 1995,
section candidate
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.398 ms) : 2350, 2447
.   : milestone, 2398,
iast (2.181 ms) : 2120, 2242
.   : milestone, 2181,
iast_GLOBAL (2.221 ms) : 2159, 2283
.   : milestone, 2221,
profiling (2.037 ms) : 1988, 2087
.   : milestone, 2037,
tracing (1.994 ms) : 1946, 2041
.   : milestone, 1994,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 2.389 ms [2.341 ms, 2.438 ms] 916.119 µs (62.2%)
iast 2.18 ms [2.119 ms, 2.241 ms] 706.823 µs (48.0%)
iast_GLOBAL 2.229 ms [2.168 ms, 2.291 ms] 755.861 µs (51.3%)
profiling 2.022 ms [1.973 ms, 2.071 ms] 548.731 µs (37.2%)
tracing 1.995 ms [1.947 ms, 2.042 ms] 521.426 µs (35.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 2.398 ms [2.35 ms, 2.447 ms] 927.82 µs (63.1%)
iast 2.181 ms [2.12 ms, 2.242 ms] 710.663 µs (48.3%)
iast_GLOBAL 2.221 ms [2.159 ms, 2.283 ms] 750.678 µs (51.1%)
profiling 2.037 ms [1.988 ms, 2.087 ms] 566.871 µs (38.6%)
tracing 1.994 ms [1.946 ms, 2.041 ms] 523.455 µs (35.6%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~99980be4b4, baseline=1.50.0-SNAPSHOT~7787af738f
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.049 s) : 15049000, 15049000
.   : milestone, 15049000,
appsec (14.908 s) : 14908000, 14908000
.   : milestone, 14908000,
iast (18.467 s) : 18467000, 18467000
.   : milestone, 18467000,
iast_GLOBAL (18.226 s) : 18226000, 18226000
.   : milestone, 18226000,
profiling (15.403 s) : 15403000, 15403000
.   : milestone, 15403000,
tracing (14.962 s) : 14962000, 14962000
.   : milestone, 14962000,
section candidate
no_agent (15.385 s) : 15385000, 15385000
.   : milestone, 15385000,
appsec (14.888 s) : 14888000, 14888000
.   : milestone, 14888000,
iast (18.678 s) : 18678000, 18678000
.   : milestone, 18678000,
iast_GLOBAL (17.905 s) : 17905000, 17905000
.   : milestone, 17905000,
profiling (15.111 s) : 15111000, 15111000
.   : milestone, 15111000,
tracing (14.924 s) : 14924000, 14924000
.   : milestone, 14924000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.049 s [15.049 s, 15.049 s] -
appsec 14.908 s [14.908 s, 14.908 s] -141.0 ms (-0.9%)
iast 18.467 s [18.467 s, 18.467 s] 3.418 s (22.7%)
iast_GLOBAL 18.226 s [18.226 s, 18.226 s] 3.177 s (21.1%)
profiling 15.403 s [15.403 s, 15.403 s] 354.0 ms (2.4%)
tracing 14.962 s [14.962 s, 14.962 s] -87.0 ms (-0.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.385 s [15.385 s, 15.385 s] -
appsec 14.888 s [14.888 s, 14.888 s] -497.0 ms (-3.2%)
iast 18.678 s [18.678 s, 18.678 s] 3.293 s (21.4%)
iast_GLOBAL 17.905 s [17.905 s, 17.905 s] 2.52 s (16.4%)
profiling 15.111 s [15.111 s, 15.111 s] -274.0 ms (-1.8%)
tracing 14.924 s [14.924 s, 14.924 s] -461.0 ms (-3.0%)

@daniel-mohedano daniel-mohedano marked this pull request as ready for review June 5, 2025 08:18
@daniel-mohedano daniel-mohedano requested review from a team as code owners June 5, 2025 08:18
nikita-tkachenko-datadog
nikita-tkachenko-datadog reviewed Jun 5, 2025
View reviewed changes
daniel-mohedano
daniel-mohedano commented Jun 6, 2025
View reviewed changes
dd-smoke-tests/backend-mock/src/main/groovy/datadog/smoketest/MockBackend.groovy
Comment on lines +56 to +63
itrEnabled = true
codeCoverageEnabled = true
testsSkippingEnabled = true
flakyRetriesEnabled = false
impactedTestsDetectionEnabled = false
knownTestsEnabled = false
testManagementEnabled = false
attemptToFixRetries = 0
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mock backend now sets each setting back to its default value before every test run, which was causing some tests to be marked with enabled settings which shouldn't be. Wasn't failing on CI because the only case of this happening just added test_management.enabled to the traces

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@datadog-datadog-prod-us1 datadog-datadog-prod-us1[bot] datadog-datadog-prod-us1[bot] left review comments

@nikita-tkachenko-datadog nikita-tkachenko-datadog nikita-tkachenko-datadog left review comments

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer PerfectSlayer is a code owner automatically assigned from DataDog/apm-java

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@daniel-mohedano @nikita-tkachenko-datadog