Skip to content

Commit f093c55

Browse files
committed
crowdstrike: clean up ingest pipeline
Use codegen for the pipeline. While doing this remove duplicated processors and group processors somewhat more logically. [git-generate] SHA=54781ceda31cbf6b74a434549c6eeb1652e0065c git clone --depth=1 [email protected]:efd6/fdr_gen.git ( cd fdr_gen git fetch --depth=1 origin ${SHA} INGEST=../packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline go run ./default.go -out ${INGEST}/default.yml go run ./categorize.go -out ${INGEST}/categorize.yml go run ./data_protection.go -out ${INGEST}/data_protection_detection_summary.yml go run ./network.go -dir inbound -out ${INGEST}/inbound_network.yml go run ./network.go -dir outbound -out ${INGEST}/outbound_network.yml ) rm -rf fdr_gen cd packages/crowdstrike elastic-package changelog add --description "Improve ingest pipeline maintainability." --type enhancement --next minor --link #1
1 parent a1e4e74 commit f093c55

File tree

7 files changed

+3075
-2308
lines changed

7 files changed

+3075
-2308
lines changed

packages/crowdstrike/changelog.yml

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,13 @@
11
# newer versions go on top
2+
- version: "2.11.0"
3+
changes:
4+
- description: Improve ingest pipeline maintainability.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/1
27
- version: "2.10.0"
38
changes:
49
- description: >-
5-
Provide an alternate endpoint to query host data for GovCloud CIDs.
6-
The GovCloud CIDs must enable the `GovCloud` flag in the integration configuration to ensure the correct endpoint is used.
10+
Provide an alternate endpoint to query host data for GovCloud CIDs. The GovCloud CIDs must enable the `GovCloud` flag in the integration configuration to ensure the correct endpoint is used.
711
type: enhancement
812
link: https://github.com/elastic/integrations/pull/16007
913
- version: "2.9.0"
@@ -62,8 +66,7 @@
6266
- version: "2.2.0"
6367
changes:
6468
- description: >-
65-
Migrate to the "/spotlight/combined/vulnerabilities/v1" endpoint for vulnerability data.
66-
Add support for the `facet` query parameter to control what data is returned in the API response.
69+
Migrate to the "/spotlight/combined/vulnerabilities/v1" endpoint for vulnerability data. Add support for the `facet` query parameter to control what data is returned in the API response.
6770
type: enhancement
6871
link: https://github.com/elastic/integrations/pull/15049
6972
- version: "2.1.0"
@@ -78,12 +81,7 @@
7881
link: https://github.com/elastic/integrations/pull/15019
7982
- version: "2.0.0"
8083
changes:
81-
- description: |
82-
Data deduplication is now disabled by default for the FDR data stream when configured with the aws-s3 input.
83-
Previously, the FDR data stream automatically handled deduplication by computing an Elasticsearch document _id
84-
using the aws-s3 input. To prevent duplicate documents, you must now explicitly enable the Data Deduplication setting.
85-
While enabling this setting prevents duplicates, it may result in a lower indexing rate because Elasticsearch
86-
must check for existing documents before indexing.
84+
- description: "Data deduplication is now disabled by default for the FDR data stream when configured with the aws-s3 input. \nPreviously, the FDR data stream automatically handled deduplication by computing an Elasticsearch document _id \nusing the aws-s3 input. To prevent duplicate documents, you must now explicitly enable the Data Deduplication setting. \nWhile enabling this setting prevents duplicates, it may result in a lower indexing rate because Elasticsearch \nmust check for existing documents before indexing.\n"
8785
type: breaking-change
8886
link: https://github.com/elastic/integrations/pull/14762
8987
- version: "1.80.0"

0 commit comments

Comments
 (0)