Commit 2927233
# Backport
This will backport the following commits from `main` to `8.19`:
- [[Attack Discovery][Scheduling] Add attack discovery alerts
deduplication (#12790)
(#223483)](#223483)
<!--- Backport version: 9.6.6 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)
<!--BACKPORT [{"author":{"name":"Ievgen
Sorokopud","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-06-13T19:35:52Z","message":"[Attack
Discovery][Scheduling] Add attack discovery alerts deduplication
(#12790) (#223483)\n\n## Summary\n\nMain ticket
([Internal\nlink](https://github.com/elastic/security-team/issues/12790))\n\nWith
these changes we add attack discovery alerts deduplication. Right\nnow
identical attack are being stored within the alerts index and to\nfilter
duplicates out we will use the next approach:\n\n1. Each alert will have
a ID generated as a SHA256 of attack attributes:\nlist of SIEM alerts
IDs and space ID\n2. After LLM generates attack discoveries we would
check whether the\nattack alert with the ID that describes the attack
based on the rule\nabove exists in the alerts index\n3. If such an
alerts exists we would drop the generated discovery\n4. Otherwise, we
would proceed with storing the attack discovery as a\nnew alert\n\n##
NOTES\n\nThe feature is hidden behind the feature flag (in
`kibana.dev.yml`):\n\n```\nfeature_flags.overrides:\n
securitySolution.attackDiscoveryAlertsEnabled:
true\n```","sha":"ed803078f99d9049ea6fcb6b7eea50ea1d3cea9d","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:
SecuritySolution","Team:Security Generative
AI","backport:version","v9.1.0","v8.19.0"],"title":"[Attack
Discovery][Scheduling] Add attack discovery alerts deduplication
(#12790)","number":223483,"url":"https://github.com/elastic/kibana/pull/223483","mergeCommit":{"message":"[Attack
Discovery][Scheduling] Add attack discovery alerts deduplication
(#12790) (#223483)\n\n## Summary\n\nMain ticket
([Internal\nlink](https://github.com/elastic/security-team/issues/12790))\n\nWith
these changes we add attack discovery alerts deduplication. Right\nnow
identical attack are being stored within the alerts index and to\nfilter
duplicates out we will use the next approach:\n\n1. Each alert will have
a ID generated as a SHA256 of attack attributes:\nlist of SIEM alerts
IDs and space ID\n2. After LLM generates attack discoveries we would
check whether the\nattack alert with the ID that describes the attack
based on the rule\nabove exists in the alerts index\n3. If such an
alerts exists we would drop the generated discovery\n4. Otherwise, we
would proceed with storing the attack discovery as a\nnew alert\n\n##
NOTES\n\nThe feature is hidden behind the feature flag (in
`kibana.dev.yml`):\n\n```\nfeature_flags.overrides:\n
securitySolution.attackDiscoveryAlertsEnabled:
true\n```","sha":"ed803078f99d9049ea6fcb6b7eea50ea1d3cea9d"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/223483","number":223483,"mergeCommit":{"message":"[Attack
Discovery][Scheduling] Add attack discovery alerts deduplication
(#12790) (#223483)\n\n## Summary\n\nMain ticket
([Internal\nlink](https://github.com/elastic/security-team/issues/12790))\n\nWith
these changes we add attack discovery alerts deduplication. Right\nnow
identical attack are being stored within the alerts index and to\nfilter
duplicates out we will use the next approach:\n\n1. Each alert will have
a ID generated as a SHA256 of attack attributes:\nlist of SIEM alerts
IDs and space ID\n2. After LLM generates attack discoveries we would
check whether the\nattack alert with the ID that describes the attack
based on the rule\nabove exists in the alerts index\n3. If such an
alerts exists we would drop the generated discovery\n4. Otherwise, we
would proceed with storing the attack discovery as a\nnew alert\n\n##
NOTES\n\nThe feature is hidden behind the feature flag (in
`kibana.dev.yml`):\n\n```\nfeature_flags.overrides:\n
securitySolution.attackDiscoveryAlertsEnabled:
true\n```","sha":"ed803078f99d9049ea6fcb6b7eea50ea1d3cea9d"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Co-authored-by: Ievgen Sorokopud <[email protected]>
1 parent c9808ad commit 2927233
File tree
18 files changed
+678
-255
lines changed- x-pack/solutions/security/plugins/elastic_assistant/server
- __mocks__
- lib/attack_discovery
- evaluation/__mocks__
- persistence
- deduplication
- get_scheduled_and_ad_hoc_index_pattern
- get_scheduled_index_pattern
- transforms/transform_to_alert_documents
- get_alert_url
- schedules/register_schedule
- routes/attack_discovery/helpers
18 files changed
+678
-255
lines changedLines changed: 2 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
| 51 | + | |
50 | 52 | | |
51 | 53 | | |
52 | 54 | | |
| |||
Lines changed: 29 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
32 | 61 | | |
Lines changed: 109 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
Lines changed: 75 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
Lines changed: 0 additions & 44 deletions
This file was deleted.
Lines changed: 0 additions & 20 deletions
This file was deleted.
Lines changed: 23 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
Lines changed: 12 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
Lines changed: 17 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
109 | 123 | | |
110 | 124 | | |
111 | 125 | | |
| |||
211 | 225 | | |
212 | 226 | | |
213 | 227 | | |
214 | | - | |
215 | | - | |
| 228 | + | |
216 | 229 | | |
217 | 230 | | |
218 | 231 | | |
| |||
336 | 349 | | |
337 | 350 | | |
338 | 351 | | |
339 | | - | |
340 | | - | |
341 | | - | |
342 | | - | |
| 352 | + | |
343 | 353 | | |
344 | 354 | | |
345 | 355 | | |
| |||
0 commit comments