[BUG] Error Creating Security Detector "Request Timeout after 30000ms"

[Aravinth31]

Description:

We are facing issues with case-insensitive matching in OpenSearch Security Analytics detectors. This GitLab issue is a continuation and deeper investigation of a problem that I previously raised on the OpenSearch forum.

The forum discussion highlights that detectors treat text fields as case-sensitive for |contains matching, which is unexpected behavior. Based on that discussion and further experimentation, we are now encountering additional issues when attempting alternative approaches to enable case-insensitive detection.

Background (Previously Raised by Me on Forum)

I previously raised the following forum issue describing the core problem:

• Detectors using |contains conditions on text fields only trigger when the case exactly matches.
• Documents with the same logical value but different casing do not trigger detections.

Example:

Rule

TargetObject|contains: \Microsoft\Terminal Server Client\Servers

Indexed document value

\REGISTRY\USER\MICROSOFT\TERMINAL SERVER CLIENT\SERVERS\Users

Despite the field being mapped as text, the detector does not trigger unless the case exactly matches, which contradicts expectations of analyzed fields.

---

Attempt 1: Case-Insensitive Regex

To address this, I updated the detection rule to use regex with a case-insensitive flag:

detection:
  selection1:
    EventType: DeleteValue
    TargetObject|re: '(?i)\\Microsoft\\Terminal Server Client\\Default\\MRU'
  selection2:
    EventType: DeleteKey
    TargetObject|re: '(?i)\\Microsoft\\Terminal Server Client\\Servers\\'
  condition: 1 of selection*

Observed Behavior

• After switching to regex, the detector started generating detections unexpectedly.
• Previously, no detections were created even for exact matches.
• With regex, detections are generated in scenarios that do not appear to strictly match the intent of the rule.

Attempt 2: Keyword Field with Normalizer

As an alternative, I changed the field mapping to use a keyword type with a lowercase normalizer:

"TargetObject": {
  "type": "keyword",
  "normalizer": "lowercase_normalizer"
}

Observed Behavior

With this mapping, detector creation consistently fails with the following error:

Request timeout after 30000ms

• There is no clear error explaining why adding a normalizer to a mapped field causes detector creation to time out.
• This makes it difficult to understand whether normalizers are supported or if there are internal constraints in the detector execution.

Expected Behavior

• Detectors should support predictable and consistent case-insensitive matching.
• |contains should not behave as strictly case-sensitive for analyzed text fields.
• Using a keyword field with a normalizer should not cause detector creation to fail silently with a timeout.
• If certain approaches are unsupported, the detector should return a clear and actionable error message.

Questions / Clarifications Needed

1. What is the expected case-sensitivity behavior for |contains and regex matching in Security Analytics detectors?
2. How are detectors internally evaluating text vs keyword fields?
3. Is there a recommended or officially supported way to implement case-insensitive detection without modifying ingested data?
4. Why does applying a normalizer to a keyword field cause detector creation to fail with a request timeout?

Reference (Forum Thread Raised by Me)

For additional background and context, please refer to the forum discussion that I originally raised:

🔗 https://forum.opensearch.org/t/opensearch-security-analytics-detector-treats-text-fields-as-case-sensitive-for-contains-matching-why-and-how-to-enable-case-insensitive-detection/27662

[andrross]

Catch All Triage - 1 2

@toepkerd @AWSHurneyt Can a maintainer take a look at this issue? Thanks!

[AWSHurneyt]

Forwarded this issue to the team manager for prioritization.