Skip to content

Conversation

@gizas
Copy link
Contributor

@gizas gizas commented Nov 6, 2025

Description

Feature: The enablement of the sending_queue in the configuration of the otplhttp exporter does not preserve the metadata information of the initial context.

This pr introduces a new configuration called metadata_keys that will be used in the exporter config

Testing

  • Added partionetr_test

  • Built my local otlp exporter

Configuration of otlphttp:

otlphttp:
        endpoint: https://endpoint.invalid/_otlp
        timeout: {{ .http_request_timeout | quote }}
        sending_queue:
          enabled: true
          num_consumers: {{ .concurrentRequests }}
          wait_for_result: false
          block_on_overflow: true  # false = data loss on queue full
          sizer: requests
          queue_size: {{ .queuedRequests }}
        metadata_keys:
          - project-id

See "ProjectID":"local" that is populated below in my logs

{"log.level":"info","@timestamp":"2025-11-06T15:51:49.955Z","message":"ecpRoutingRoundTripper: after URL modification","resource":{"cloud.availability_zone":"local","k8s.namespace.name":"motel-index-collector","k8s.node.name":"hotel-worker2","k8s.pod.name":"motel-index-collector-local-58484b67c8-l4vrt","k8s.pod.uid":"1a8f6fad-91b6-413a-a929-6d004ca489b3","orchestrator.cluster.name":"default","orchestrator.deploymentslice":"","orchestrator.environment":"default","service.instance.id":"ecf43bce-5aac-444c-b244-8871bbf2d69a","service.name":"motel-index-collector","service.version":"git"},"otelcol.component.id":"ecproutingmiddleware","otelcol.component.kind":"extension","final_url_host":"local.es.svc.cluster.local:9200","final_url_scheme":"http","final_url":"[http://local.es.svc.cluster.local:9200/_otlp/v1/metrics","final_host":"","ProjectID":"local","header_count":6,"ecs.version":"1.6.0"}](http://local.es.svc.cluster.local:9200/_otlp/v1/metrics%22,%22final_host%22:%22%22,%22ProjectID%22:%22local%22,%22header_count%22:6,%22ecs.version%22:%221.6.0%22%7D)

gizas added 3 commits November 6, 2025 16:07
Signed-off-by: Andreas Gkizas <[email protected]>
Signed-off-by: Andreas Gkizas <[email protected]>
@gizas gizas requested a review from a team as a code owner November 6, 2025 15:56
@gizas gizas requested a review from bogdandrutu November 6, 2025 15:56
Signed-off-by: Andreas Gkizas <[email protected]>
@codecov
Copy link

codecov bot commented Nov 6, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.25%. Comparing base (a6a1766) to head (00a18f3).

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #14139   +/-   ##
=======================================
  Coverage   92.25%   92.25%           
=======================================
  Files         657      658    +1     
  Lines       41068    41115   +47     
=======================================
+ Hits        37888    37932   +44     
- Misses       2176     2178    +2     
- Partials     1004     1005    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

gizas added 2 commits November 6, 2025 18:25
Signed-off-by: Andreas Gkizas <[email protected]>
@gizas
Copy link
Contributor Author

gizas commented Nov 13, 2025

@bogdandrutu (sorry for direct ping) did you by any chance have time to review above pr? Do I have to add someone else specific?

Any hint will be really appreciated

// The encoding to export telemetry (default: "proto")
Encoding EncodingType `mapstructure:"encoding"`

// MetadataKeys is a list of metadata keys that will be used to partition the datainto batches if sending_queue is enabled
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be in exporterhelper.QueueBatchConfig, so (a) it's kept close to the related batching config, and (b) it's available to all exporters?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants