AAP service activation origin metric #8934
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
from
437276f to
03e4246
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 7 performance improvements and 1 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~67186fc538, baseline=1.50.0-SNAPSHOT~7787af738f
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.024 s) : 0, 1024366
Total [baseline] (8.555 s) : 0, 8554737
Agent [candidate] (992.474 ms) : 0, 992474
Total [candidate] (8.531 s) : 0, 8530501
section iast
Agent [baseline] (1.154 s) : 0, 1153865
Total [baseline] (9.16 s) : 0, 9159774
Agent [candidate] (1.124 s) : 0, 1124247
Total [candidate] (9.212 s) : 0, 9212421
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.16 s) : 0, 1159939
Total [baseline] (9.155 s) : 0, 9154866
Agent [candidate] (1.125 s) : 0, 1124733
Total [candidate] (9.226 s) : 0, 9226338
section iast_TELEMETRY_OFF
Agent [baseline] (1.148 s) : 0, 1148350
Total [baseline] (9.233 s) : 0, 9233192
Agent [candidate] (1.122 s) : 0, 1122125
Total [candidate] (9.22 s) : 0, 9219593
gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~67186fc538, baseline=1.50.0-SNAPSHOT~7787af738f
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.641 ms) : 0, 684641
BytebuddyAgent [candidate] (684.898 ms) : 0, 684898
GlobalTracer [baseline] (240.816 ms) : 0, 240816
GlobalTracer [candidate] (239.558 ms) : 0, 239558
AppSec [baseline] (57.369 ms) : 0, 57369
AppSec [candidate] (29.378 ms) : 0, 29378
Debugger [baseline] (6.233 ms) : 0, 6233
Debugger [candidate] (6.064 ms) : 0, 6064
Remote Config [baseline] (757.303 µs) : 0, 757
Remote Config [candidate] (649.774 µs) : 0, 650
Telemetry [baseline] (10.812 ms) : 0, 10812
Telemetry [candidate] (8.274 ms) : 0, 8274
section iast
BytebuddyAgent [baseline] (804.842 ms) : 0, 804842
BytebuddyAgent [candidate] (800.645 ms) : 0, 800645
GlobalTracer [baseline] (230.781 ms) : 0, 230781
GlobalTracer [candidate] (230.157 ms) : 0, 230157
IAST [baseline] (27.944 ms) : 0, 27944
IAST [candidate] (25.341 ms) : 0, 25341
AppSec [baseline] (52.125 ms) : 0, 52125
AppSec [candidate] (30.329 ms) : 0, 30329
Debugger [baseline] (6.016 ms) : 0, 6016
Debugger [candidate] (5.823 ms) : 0, 5823
Remote Config [baseline] (604.288 µs) : 0, 604
Remote Config [candidate] (583.436 µs) : 0, 583
Telemetry [baseline] (7.948 ms) : 0, 7948
Telemetry [candidate] (7.897 ms) : 0, 7897
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (809.197 ms) : 0, 809197
BytebuddyAgent [candidate] (801.044 ms) : 0, 801044
GlobalTracer [baseline] (232.215 ms) : 0, 232215
GlobalTracer [candidate] (230.474 ms) : 0, 230474
IAST [baseline] (28.213 ms) : 0, 28213
IAST [candidate] (28.722 ms) : 0, 28722
AppSec [baseline] (51.993 ms) : 0, 51993
AppSec [candidate] (26.759 ms) : 0, 26759
Debugger [baseline] (5.998 ms) : 0, 5998
Debugger [candidate] (5.844 ms) : 0, 5844
Remote Config [baseline] (607.909 µs) : 0, 608
Remote Config [candidate] (592.015 µs) : 0, 592
Telemetry [baseline] (7.98 ms) : 0, 7980
Telemetry [candidate] (7.861 ms) : 0, 7861
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (800.04 ms) : 0, 800040
BytebuddyAgent [candidate] (798.873 ms) : 0, 798873
GlobalTracer [baseline] (231.158 ms) : 0, 231158
GlobalTracer [candidate] (230.617 ms) : 0, 230617
IAST [baseline] (29.662 ms) : 0, 29662
IAST [candidate] (28.321 ms) : 0, 28321
AppSec [baseline] (49.417 ms) : 0, 49417
AppSec [candidate] (26.631 ms) : 0, 26631
Debugger [baseline] (5.992 ms) : 0, 5992
Debugger [candidate] (5.805 ms) : 0, 5805
Remote Config [baseline] (607.88 µs) : 0, 608
Remote Config [candidate] (587.93 µs) : 0, 588
Telemetry [baseline] (7.896 ms) : 0, 7896
Telemetry [candidate] (7.802 ms) : 0, 7802
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~67186fc538, baseline=1.50.0-SNAPSHOT~7787af738f
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.02 s) : 0, 1020288
Total [baseline] (11.11 s) : 0, 11109589
Agent [candidate] (998.654 ms) : 0, 998654
Total [candidate] (11.058 s) : 0, 11057845
section appsec
Agent [baseline] (1.172 s) : 0, 1171580
Total [baseline] (11.251 s) : 0, 11251133
Agent [candidate] (1.156 s) : 0, 1155584
Total [candidate] (11.351 s) : 0, 11350816
section iast
Agent [baseline] (1.149 s) : 0, 1149391
Total [baseline] (11.308 s) : 0, 11308306
Agent [candidate] (1.142 s) : 0, 1142364
Total [candidate] (11.415 s) : 0, 11414966
section profiling
Agent [baseline] (1.263 s) : 0, 1262849
Total [baseline] (11.442 s) : 0, 11442411
Agent [candidate] (1.241 s) : 0, 1241481
Total [candidate] (11.46 s) : 0, 11459561
gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~67186fc538, baseline=1.50.0-SNAPSHOT~7787af738f
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (681.204 ms) : 0, 681204
BytebuddyAgent [candidate] (688.377 ms) : 0, 688377
GlobalTracer [baseline] (239.607 ms) : 0, 239607
GlobalTracer [candidate] (241.602 ms) : 0, 241602
AppSec [baseline] (55.171 ms) : 0, 55171
AppSec [candidate] (29.739 ms) : 0, 29739
Debugger [baseline] (6.239 ms) : 0, 6239
Debugger [candidate] (6.154 ms) : 0, 6154
Remote Config [baseline] (734.685 µs) : 0, 735
Remote Config [candidate] (663.588 µs) : 0, 664
Telemetry [baseline] (13.666 ms) : 0, 13666
Telemetry [candidate] (8.341 ms) : 0, 8341
section appsec
BytebuddyAgent [baseline] (706.679 ms) : 0, 706679
BytebuddyAgent [candidate] (701.974 ms) : 0, 701974
GlobalTracer [baseline] (239.346 ms) : 0, 239346
GlobalTracer [candidate] (238.083 ms) : 0, 238083
IAST [baseline] (22.1 ms) : 0, 22100
IAST [candidate] (22.05 ms) : 0, 22050
AppSec [baseline] (176.658 ms) : 0, 176658
AppSec [candidate] (163.826 ms) : 0, 163826
Debugger [baseline] (6.003 ms) : 0, 6003
Debugger [candidate] (5.98 ms) : 0, 5980
Remote Config [baseline] (636.704 µs) : 0, 637
Remote Config [candidate] (633.583 µs) : 0, 634
Telemetry [baseline] (7.411 ms) : 0, 7411
Telemetry [candidate] (10.501 ms) : 0, 10501
section iast
BytebuddyAgent [baseline] (800.306 ms) : 0, 800306
BytebuddyAgent [candidate] (814.554 ms) : 0, 814554
GlobalTracer [baseline] (231.482 ms) : 0, 231482
GlobalTracer [candidate] (233.513 ms) : 0, 233513
IAST [baseline] (26.07 ms) : 0, 26070
IAST [candidate] (28.077 ms) : 0, 28077
AppSec [baseline] (53.492 ms) : 0, 53492
AppSec [candidate] (28.041 ms) : 0, 28041
Debugger [baseline] (5.959 ms) : 0, 5959
Debugger [candidate] (5.901 ms) : 0, 5901
Remote Config [baseline] (603.288 µs) : 0, 603
Remote Config [candidate] (591.938 µs) : 0, 592
Telemetry [baseline] (7.954 ms) : 0, 7954
Telemetry [candidate] (7.881 ms) : 0, 7881
section profiling
BytebuddyAgent [baseline] (674.404 ms) : 0, 674404
BytebuddyAgent [candidate] (675.833 ms) : 0, 675833
GlobalTracer [baseline] (359.495 ms) : 0, 359495
GlobalTracer [candidate] (361.326 ms) : 0, 361326
AppSec [baseline] (61.394 ms) : 0, 61394
AppSec [candidate] (30.553 ms) : 0, 30553
Debugger [baseline] (6.027 ms) : 0, 6027
Debugger [candidate] (11.934 ms) : 0, 11934
Remote Config [baseline] (643.648 µs) : 0, 644
Remote Config [candidate] (697.336 µs) : 0, 697
Telemetry [baseline] (8.024 ms) : 0, 8024
Telemetry [candidate] (8.993 ms) : 0, 8993
ProfilingAgent [baseline] (102.166 ms) : 0, 102166
ProfilingAgent [candidate] (101.164 ms) : 0, 101164
Profiling [baseline] (102.19 ms) : 0, 102190
Profiling [candidate] (101.188 ms) : 0, 101188
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 17 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~67186fc538, baseline=1.50.0-SNAPSHOT~7787af738f
dateFormat X
axisFormat %s
section baseline
no_agent (13.339 ms) : 13201, 13477
. : milestone, 13339,
appsec (78.643 ms) : 77468, 79819
. : milestone, 78643,
appsec_no_iast (87.899 ms) : 86522, 89275
. : milestone, 87899,
code_origins (104.08 ms) : 101740, 106420
. : milestone, 104080,
iast (95.601 ms) : 93893, 97310
. : milestone, 95601,
profiling (69.261 ms) : 68254, 70269
. : milestone, 69261,
tracing (58.631 ms) : 57720, 59542
. : milestone, 58631,
section candidate
no_agent (13.272 ms) : 13139, 13406
. : milestone, 13272,
appsec (76.258 ms) : 75233, 77282
. : milestone, 76258,
appsec_no_iast (84.881 ms) : 83623, 86138
. : milestone, 84881,
code_origins (100.983 ms) : 98782, 103184
. : milestone, 100983,
iast (92.792 ms) : 91307, 94277
. : milestone, 92792,
profiling (67.063 ms) : 66106, 68019
. : milestone, 67063,
tracing (56.537 ms) : 55668, 57406
. : milestone, 56537,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~67186fc538, baseline=1.50.0-SNAPSHOT~7787af738f
dateFormat X
axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
. : milestone, 1473,
appsec (2.406 ms) : 2358, 2455
. : milestone, 2406,
iast (2.188 ms) : 2127, 2250
. : milestone, 2188,
iast_GLOBAL (2.227 ms) : 2166, 2289
. : milestone, 2227,
profiling (2.024 ms) : 1975, 2073
. : milestone, 2024,
tracing (2.002 ms) : 1955, 2050
. : milestone, 2002,
section candidate
no_agent (1.474 ms) : 1462, 1486
. : milestone, 1474,
appsec (2.412 ms) : 2363, 2461
. : milestone, 2412,
iast (2.189 ms) : 2128, 2250
. : milestone, 2189,
iast_GLOBAL (2.229 ms) : 2168, 2291
. : milestone, 2229,
profiling (2.032 ms) : 1982, 2081
. : milestone, 2032,
tracing (2.004 ms) : 1957, 2052
. : milestone, 2004,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~67186fc538, baseline=1.50.0-SNAPSHOT~7787af738f
dateFormat X
axisFormat %s
section baseline
no_agent (14.945 s) : 14945000, 14945000
. : milestone, 14945000,
appsec (14.871 s) : 14871000, 14871000
. : milestone, 14871000,
iast (18.838 s) : 18838000, 18838000
. : milestone, 18838000,
iast_GLOBAL (18.086 s) : 18086000, 18086000
. : milestone, 18086000,
profiling (15.112 s) : 15112000, 15112000
. : milestone, 15112000,
tracing (14.912 s) : 14912000, 14912000
. : milestone, 14912000,
section candidate
no_agent (15.5 s) : 15500000, 15500000
. : milestone, 15500000,
appsec (14.799 s) : 14799000, 14799000
. : milestone, 14799000,
iast (18.302 s) : 18302000, 18302000
. : milestone, 18302000,
iast_GLOBAL (18.196 s) : 18196000, 18196000
. : milestone, 18196000,
profiling (15.416 s) : 15416000, 15416000
. : milestone, 15416000,
tracing (15.071 s) : 15071000, 15071000
. : milestone, 15071000,
|
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
5 times, most recently
from
257c509 to
a36069c
Compare
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
2 times, most recently
from
7e2e5e9 to
d353cbd
Compare
Signed-off-by: sezen.leblay <[email protected]>
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
from
20f69bd to
7b3cf77
Compare
sezen-datadog
requested review from
smola,
manuel-alvarez-alvarez and
dougqh
and removed request for
a team
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
| import java.util.concurrent.ArrayBlockingQueue; | ||
| import java.util.concurrent.BlockingQueue; | ||
|
|
||
| public class AppSecMetricCollector implements MetricCollector<MetricCollector.Metric> { |
There was a problem hiding this comment.
Traditionally we've used the WafMetricCollector as the source for all appsec metrics, although I'm not against about this new one.
| // Periodically report AppSec enabled status | ||
| final ProductActivation appSecActivation = Config.get().getAppSecActivation(); | ||
| if (appSecActivation == ProductActivation.FULLY_ENABLED | ||
| || ConfigOrigin.REMOTE == appsecOrigin) { // remote enablement can happen later |
There was a problem hiding this comment.
If I understood correctly, you have to report the metric in every heartbeat when appsec is fully enabled (not only when it's configured via RC),
There was a problem hiding this comment.
yes, but we have to report remote enablement as well
| @@ -521,6 +523,11 @@ private void setAppSecActivation(final AppSecFeatures.Asm asm) { | |||
| newState = tracerConfig.getAppSecActivation() == ProductActivation.FULLY_ENABLED; | |||
There was a problem hiding this comment.
If we receive a null via RC then we should go back to report the metric with the locally configured origin (default, env_var, ...)
| if (newState) { | ||
| AppSecMetricCollector.appsecOrigin = ConfigOrigin.REMOTE; | ||
| } else { | ||
| AppSecMetricCollector.appsecOrigin = null; |
There was a problem hiding this comment.
In here the source is still RC, the difference is that appsec is disabled so we should not report the gauge
sezen-datadog
changed the base branch from
master
to
sezen.leblay/upgrade-libddwaf-java-1.23
sezen-datadog
force-pushed
the
sezen.leblay/upgrade-libddwaf-java-1.23
branch
from
6c47b1e to
42f13c9
Compare
sezen-datadog
force-pushed
the
sezen.leblay/APPSEC-57893-appsec-enabled-metric
branch
from
8f3cc32 to
67186fc
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
It sends appsec.enabled metric every 60 seconds (existing heartbeat value for telemetry)
Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: APPSEC-57893