[connector/elasticapm] Add ability configure metric aggregation limit configs

[isaacaflores2]

Exposed aggregation limit configs that are propagated to the lsmintervalprocessor.

Note: I opted to keep the configuration names ( resource_limit ) the same as the lsmintervalprocessor mostly so they are consistent for both components. Another option would be to restructure everything to omit the _limit suffix so elasticapm::aggregation::limit::resource and elasticapm::aggregation::limit::scope. Let me know what you think.

Checklist

• Update connector/elasticapmconnector/README.md to includes the limits in the example config.yml
• Add test cases to assert overflow behavior
• Run collector locally to verify overflow behavior
• Double check the default values are appropriate and match MIS
  • I found the default values under apm-managed-service/checkout/apm-aggregation-service/config/config.go

                        // Defaults to apm-server 8GB equivalent limits.
   		MaxSpanGroups:                         40000,
   		MaxSpanGroupsPerService:               4000,
   		MaxTransactionGroups:                  40000,
   		MaxTransactionGroupsPerService:        4000,
   		MaxServiceTransactionGroups:           8000,
   		MaxServiceTransactionGroupsPerService: 800,
   		MaxServices:                           8000,

Local Testing

1. Build collector make genelasticcol
2. Run the collector with the below config

receivers:
 otlp:
   protocols:
     grpc:
       endpoint: 0.0.0.0:4317
       include_metadata: true
     http:
       endpoint: 0.0.0.0:4318
       include_metadata: true

exporters:
 otlp:
   endpoint: "${env:ELASTIC_APM_SERVER_URL:-localhost:8200}"
   headers:
     Authorization: "Bearer ${env:ELASTIC_APM_TOKEN}"
   sending_queue:
     enabled: false
   timeout: 60s
 debug:

connectors:
 elasticapm:
   aggregation:
     resource_limit:
       max_cardinality: 1
       overflow:
         attributes:
           - key: "overflow"
             value: true
     scope_limit:
       max_cardinality: 1
       overflow:
         attributes:
           - key: "overflow"
             value: true
     metric_limit:
       max_cardinality: 1
       overflow:
         attributes:
           - key: "overflow"
             value: true
     datapoint_limit:
       max_cardinality: 1
       overflow:
         attributes:
           - key: "overflow"
             value: true

service:
 pipelines:
   logs:
     receivers: [otlp]
     exporters: [elasticapm, otlp]
   metrics:
     receivers: [otlp]
     exporters: [elasticapm, otlp]
   traces:
     receivers: [otlp]
     exporters: [elasticapm, otlp]
   metrics/apm:
     receivers: [elasticapm]
     exporters: [otlp, debug]

3. Send data with different service names (local_test_1, local_test_2)

• telemetrygen traces --duration 1s --otlp-insecure --service "local_test_1"

4. Search for overflow metric documents with query labels.overflow:true to validate overflow behavior

   Example Overflow metric

   {
     "_index": ".ds-metrics-apm.service_summary.1m-default-2025.06.23-000001",
     "_id": "WTjqnpcB1DgWoZRojKGO",
     "_version": 1,
     "_source": {
       "observer": {
         "hostname": "1bc7656c85ce",
         "type": "apm-server",
         "version": "9.0.0"
       },
       "agent": {
         "name": "otlp"
       },
       "@timestamp": "2025-06-23T22:22:00.000Z",
       "data_stream": {
         "namespace": "default",
         "type": "metrics",
         "dataset": "apm.service_summary.1m"
       },
       "service": {
         "name": "unknown",
         "language": {
           "name": "unknown"
         }
       },
       "metricset": {
         "name": "service_summary",
         "interval": "1m"
       },
       "event": {},
       "labels": {
         "overflow": "true",
         "user": "isaac.flores",
         "local": "test"
       }
     },
     "fields": {
       "service.name": [
         "unknown"
       ],
       "data_stream.namespace": [
         "default"
       ],
       "service.language.name.text": [
         "unknown"
       ],
       "metricset.interval": [
         "1m"
       ],
       "data_stream.type": [
         "metrics"
       ],
       "observer.hostname": [
         "1bc7656c85ce"
       ],
       "service.language.name": [
         "unknown"
       ],
       "labels.user": [
         "isaac.flores"
       ],
       "metricset.name": [
         "service_summary"
       ],
       "@timestamp": [
         "2025-06-23T22:22:00.000Z"
       ],
       "agent.name.text": [
         "otlp"
       ],
       "observer.type": [
         "apm-server"
       ],
       "observer.version": [
         "9.0.0"
       ],
       "service.name.text": [
         "unknown"
       ],
       "data_stream.dataset": [
         "apm.service_summary.1m"
       ],
       "processor.event": [
         "metric"
       ],
       "agent.name": [
         "otlp"
       ],
       "labels.local": [
         "test"
       ],
       "labels.overflow": [
         "true"
       ]
     }
   }

Reference

Closes : #465

[isaacaflores2]

Note: I was thinking we could probably not expose these attributes for this component, so a user just defines the max cardinality for each limit. Or at least define some defaults such as:

overflow:
        attributes:
          - key: "overflow"
            value: true
          - key: "component"
             value: "connector/elasticapmconnector"

[isaacaflores2]

I added default overflow attribute values