[Enhancement] Add Entropy Enrichments to PowerShell events by w0rk3r · Pull Request #15698 · elastic/integrations

w0rk3r · 2025-10-20T17:46:57Z

Proposed commit message

Adds powershell.file.script_block_entropy_bits and powershell.file.script_block_surprisal_sd
fields computed at ingest time using Shannon entropy over Unicode characters. These fields
quantify character-level randomness and distribution uniformity of PowerShell script blocks
to provide context for obfuscation detection. Also adds script_block_length and
script_block_unique_symbols for enrichment analysis.

Summary

Related to https://github.com/elastic/ia-trade-team/issues/704

Adds four fields for script analysis at ingest time:

script_block_entropy_bits: Shannon entropy (0-20 bits) measuring randomness
script_block_surprisal_sd: Distribution uniformity of characters
script_block_length: Total character count
script_block_unique_symbols: Distinct character count

These fields help us improve and tune PowerShell obfuscation detections by providing additional context about the scripts content.

Inspired by: Painless Parsing: Shannon Entropy Calculation - Services

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

elasticmachine · 2025-10-20T20:00:31Z

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

efd6

Note that you can also calculate the entropy variance with very little extra work.

packages/windows/data_stream/forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml

elastic-vault-github-plugin-prod · 2025-10-28T19:02:35Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

packages/windows/data_stream/powershell_operational/fields/fields.yml

packages/windows/docs/README.md

packages/windows/data_stream/powershell_operational/elasticsearch/ingest_pipeline/default.yml

efd6

LGTM after nits (but not a code owner).

packages/windows/data_stream/forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml

…peline/powershell_operational.yml Co-authored-by: Dan Kortschak <[email protected]>

elasticmachine · 2025-10-29T07:41:08Z

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

…peline/powershell_operational.yml Co-authored-by: Dan Kortschak <[email protected]>

packages/windows/data_stream/forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml

…peline/powershell_operational.yml

packages/windows/data_stream/powershell_operational/elasticsearch/ingest_pipeline/default.yml

elasticmachine · 2025-10-29T17:17:02Z

💚 Build Succeeded

Buildkite Build
Commit: 6d331b7

History

💚 Build #33349 succeeded 2052ec7
💚 Build #33345 succeeded a2f002a
💚 Build #33319 succeeded 55404d4
💔 Build #33317 failed b339352
💔 Build #33316 failed 13c32b3
💔 Build #33315 failed 8a817d2

cc @w0rk3r

elastic-vault-github-plugin-prod · 2025-11-06T21:39:51Z

Package windows - 3.1.3 containing this change is available at https://epr.elastic.co/package/windows/3.1.3/

…5698) * [Enhancement] Add Script Entropy Fields to PowerShell events * Update default.yml * Drop normalized entropy * Test * v2 * v2 - forwarded data stream * Apply suggestions * Update packages/windows/data_stream/forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml Co-authored-by: Dan Kortschak <[email protected]> * Update README.md * update expected json files - script_block_surprisal_stdev * Update packages/windows/data_stream/forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml Co-authored-by: Dan Kortschak <[email protected]> * Update packages/windows/data_stream/forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml * Update default.yml --------- Co-authored-by: Dan Kortschak <[email protected]>

[Enhancement] Add Script Entropy Fields to PowerShell events

9e5d642

w0rk3r self-assigned this Oct 20, 2025

w0rk3r requested a review from a team as a code owner October 20, 2025 17:46

w0rk3r added the enhancement New feature or request label Oct 20, 2025

w0rk3r requested a review from a team as a code owner October 20, 2025 17:46

w0rk3r added the Integration:windows Windows label Oct 20, 2025

w0rk3r requested review from a team, faec and rdner October 20, 2025 17:46

andrewkroh added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. Team:Security-Windows Platform Security Windows Platform team [elastic/sec-windows-platform] labels Oct 20, 2025

efd6 reviewed Oct 20, 2025

View reviewed changes

packages/windows/data_stream/forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml Outdated Show resolved Hide resolved

w0rk3r marked this pull request as draft October 20, 2025 21:02

w0rk3r added 5 commits October 20, 2025 18:03

Update default.yml

d888514

Drop normalized entropy

623eb8c

Test

c239d0a

Merge branch 'main' into posh_entropy

c561f26

v2

223f784

w0rk3r changed the title ~~[Enhancement] Add Script Entropy Fields to PowerShell events~~ [Enhancement] Add Entropy Enrichments to PowerShell events Oct 28, 2025

w0rk3r added 2 commits October 28, 2025 13:54

Merge branch 'main' into posh_entropy

0a539be

v2 - forwarded data stream

129864c

w0rk3r marked this pull request as ready for review October 28, 2025 19:14

w0rk3r requested a review from efd6 October 28, 2025 19:14

efd6 reviewed Oct 28, 2025

View reviewed changes

Apply suggestions

8a817d2

w0rk3r requested a review from efd6 October 28, 2025 22:19

efd6 approved these changes Oct 28, 2025

View reviewed changes

Update packages/windows/data_stream/forwarded/elasticsearch/ingest_pi…

13c32b3

…peline/powershell_operational.yml Co-authored-by: Dan Kortschak <[email protected]>

w0rk3r added 2 commits October 28, 2025 20:14

Update README.md

b339352

update expected json files - script_block_surprisal_stdev

55404d4

pierrehilbert added the Team:Elastic-Agent-Data-Plane Agent Data Plane team [elastic/elastic-agent-data-plane] label Oct 29, 2025

Update packages/windows/data_stream/forwarded/elasticsearch/ingest_pi…

618da7e

…peline/powershell_operational.yml Co-authored-by: Dan Kortschak <[email protected]>

w0rk3r commented Oct 29, 2025

View reviewed changes

packages/windows/data_stream/forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml Outdated Show resolved Hide resolved

Update packages/windows/data_stream/forwarded/elasticsearch/ingest_pi…

a2f002a

…peline/powershell_operational.yml

efd6 reviewed Oct 29, 2025

View reviewed changes

packages/windows/data_stream/powershell_operational/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved

Update default.yml

2052ec7

faec approved these changes Oct 29, 2025

View reviewed changes

Merge branch 'main' into posh_entropy

6d331b7

nfritts approved these changes Nov 6, 2025

View reviewed changes

w0rk3r merged commit 4bbefe6 into main Nov 6, 2025
7 checks passed

w0rk3r deleted the posh_entropy branch November 6, 2025 19:12

w0rk3r mentioned this pull request Nov 7, 2025

[BugFix] PowerShell - Fix user.name, user.domain, and script_block_signature #15834

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Enhancement] Add Entropy Enrichments to PowerShell events#15698

[Enhancement] Add Entropy Enrichments to PowerShell events#15698
w0rk3r merged 16 commits intomainfrom
posh_entropy

w0rk3r commented Oct 20, 2025 •

edited

Loading

Uh oh!

elasticmachine commented Oct 20, 2025

Uh oh!

efd6 left a comment

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Oct 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

efd6 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Oct 29, 2025

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Oct 29, 2025

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

w0rk3r commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Summary

Checklist

Uh oh!

elasticmachine commented Oct 20, 2025

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Oct 28, 2025

🚀 Benchmarks report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Oct 29, 2025

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Oct 29, 2025

💚 Build Succeeded

History

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

w0rk3r commented Oct 20, 2025 •

edited

Loading