[azure logs] Add support for body.structured logs events from azurelogs encoding extension (OTel)

[zmoog]

Proposed commit message

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Screenshots

[zmoog]

Quick test using the Dev Tools:

Send a ResourceHealth log event as a JSON string inside message field:

POST logs-azure.events-default/_doc
{
  "@timestamp": "2025-11-06T21:44:43+01:00",
  "message": """{"category":"ResourceHealth","correlationId":"1c867fe2-050c-4a74-bb1c-a83b15246fdd","level":"Information","operationName":"Microsoft.Resourcehealth/healthevent/Updated/action","properties":{"eventCategory":"ResourceHealth","eventProperties":{"cause":"PlatformInitiated"}},"resourceId":"/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.domainRegistration","resultType":"Updated","time":"2021-05-25T22:04:07.22Z"}"""
}

Send a ResourceHealth log event as an object inside body.structured field:

POST logs-azure.events-default/_doc
{
  "@timestamp": "2025-11-06T21:44:43+01:00",
  "body": {
    "structured": {
      "category": "ResourceHealth",
      "correlationId": "1c867fe2-050c-4a74-bb1c-a83b15246fdd",
      "level": "Information",
      "operationName": "Microsoft.Resourcehealth/healthevent/Updated/action",
      "properties": {
        "eventCategory": "ResourceHealth",
        "eventProperties": {
          "cause": "PlatformInitiated"
        }
      },
      "resourceId": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.domainRegistration",
      "resultType": "Updated",
      "time": "2021-05-25T22:04:07.22Z"
    }
  }
}

GET logs-azure.activitylogs-default/_search

JSON string inside `message` field

{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1,
        "hits": [
            {
                "_index": ".ds-logs-azure.activitylogs-default-2025.11.06-000001",
                "_id": "OTH6WpoBGriDMOmNWcJf",
                "_score": 1,
                "_source": {
                    "cloud": {
                        "provider": "azure"
                    },
                    "@timestamp": "2021-05-25T22:04:07.220Z",
                    "ecs": {
                        "version": "8.11.0"
                    },
                    "related": {
                        "entity": [
                            "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.domainRegistration"
                        ]
                    },
                    "log": {
                        "level": "Information"
                    },
                    "data_stream": {
                        "namespace": "default",
                        "type": "logs",
                        "dataset": "azure.activitylogs"
                    },
                    "event": {
                        "agent_id_status": "missing",
                        "ingested": "2025-11-06T21:02:27Z",
                        "kind": "event",
                        "action": "Microsoft.Resourcehealth/healthevent/Updated/action",
                        "dataset": "azure.activitylogs"
                    },
                    "azure": {
                        "subscription_id": "00000000-0000-0000-0000-000000000000",
                        "resource": {
                            "provider": "Microsoft.domainRegistration",
                            "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.domainRegistration"
                        },
                        "correlation_id": "1c867fe2-050c-4a74-bb1c-a83b15246fdd",
                        "activitylogs": {
                            "operation_name": "Microsoft.Resourcehealth/healthevent/Updated/action",
                            "result_type": "Updated",
                            "category": "ResourceHealth",
                            "event_category": "ResourceHealth",
                            "properties": {
                                "eventProperties": {
                                    "cause": "PlatformInitiated"
                                }
                            }
                        }
                    }
                }
            }
        ]
    }
}

object inside `body.structured` field

{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1,
        "hits": [
            {
                "_index": ".ds-logs-azure.activitylogs-default-2025.11.06-000001",
                "_id": "bzH9WpoBGriDMOmN3MIt",
                "_score": 1,
                "_source": {
                    "cloud": {
                        "provider": "azure"
                    },
                    "@timestamp": "2021-05-25T22:04:07.220Z",
                    "ecs": {
                        "version": "8.11.0"
                    },
                    "related": {
                        "entity": [
                            "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.domainRegistration"
                        ]
                    },
                    "log": {
                        "level": "Information"
                    },
                    "data_stream": {
                        "namespace": "default",
                        "type": "logs",
                        "dataset": "azure.activitylogs"
                    },
                    "event": {
                        "agent_id_status": "missing",
                        "ingested": "2025-11-06T21:06:17Z",
                        "kind": "event",
                        "action": "Microsoft.Resourcehealth/healthevent/Updated/action",
                        "dataset": "azure.activitylogs"
                    },
                    "azure": {
                        "subscription_id": "00000000-0000-0000-0000-000000000000",
                        "resource": {
                            "provider": "Microsoft.domainRegistration",
                            "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.domainRegistration"
                        },
                        "correlation_id": "1c867fe2-050c-4a74-bb1c-a83b15246fdd",
                        "activitylogs": {
                            "operation_name": "Microsoft.Resourcehealth/healthevent/Updated/action",
                            "result_type": "Updated",
                            "category": "ResourceHealth",
                            "event_category": "ResourceHealth",
                            "properties": {
                                "eventProperties": {
                                    "cause": "PlatformInitiated"
                                }
                            }
                        }
                    }
                }
            }
        ]
    }
}

The search, save the result in two files, and compare them:

diff message.json body.structured.json
19c19
<                 "_id": "OTH6WpoBGriDMOmNWcJf",
---
>                 "_id": "bzH9WpoBGriDMOmN3MIt",
44c44
<                         "ingested": "2025-11-06T21:02:27Z",
---
>                         "ingested": "2025-11-06T21:06:17Z",
``

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[kaiyan-sheng]

The result comparison looks great! I assume we have a separate PR for routing documents from logs-azure.resourcelogs-default to the correct data streams based on log type?

[zmoog]

The result comparison looks great! I assume we have a separate PR for routing documents from logs-azure.resourcelogs-default to the correct data streams based on log type?

Yep, users can route log events encoded by the azurelogs translator to the Azure Logs integration using something like #15953

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 795d383

History

• 💚 Build #33791 succeeded 1d6ca2f

cc @zmoog

[zmoog]

We decided not to move forward with this implementation.