Skip to content

Prometheus exponential histogram validate scale range #6779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
dashpole opened this issue May 15, 2025 · 1 comment · May be fixed by #6822
Open

Prometheus exponential histogram validate scale range #6779

dashpole opened this issue May 15, 2025 · 1 comment · May be fixed by #6822
Assignees
@highlyavailable
Labels
good first issue Good for newcomers help wanted Extra attention is needed
Milestone
v1.38.0

Comments

@dashpole
Copy link
Contributor

I'm doing a late review of #6421, and am comparing it to the implementation here: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/9cb3ada6449ae195aa904f82f1ac972ff401974b/pkg/translator/prometheusremotewrite/histograms.go#L50

We need to ensure the scale is valid for Prometheus (-4 to 8), and scale down if it is greater than 8, or reject.

cc @GiedriusS
@MrAlias MrAlias added help wanted Extra attention is needed good first issue Good for newcomers labels May 22, 2025
@MrAlias MrAlias added this to the v1.37.0 milestone May 22, 2025
@highlyavailable
Copy link
Contributor

@MrAlias could I pick this up?

highlyavailable added a commit to highlyavailable/opentelemetry-go that referenced this issue May 24, 2025
@highlyavailable
prometheus: validate exponential histogram scale range (open-telemetr…
e90f741 
…y#6779)
@highlyavailable highlyavailable mentioned this issue May 24, 2025
Closed
highlyavailable added a commit to highlyavailable/opentelemetry-go that referenced this issue May 24, 2025
@highlyavailable
prometheus: validate exponential histogram scale range (open-telemetr…
6f10f59 
…y#6779)
@highlyavailable highlyavailable linked a pull request May 24, 2025 that will close this issue
Open
highlyavailable added a commit to highlyavailable/opentelemetry-go that referenced this issue May 24, 2025
@highlyavailable
prometheus: validate exponential histogram scale range (open-telemetr…
4c1e41b 
…y#6779)
highlyavailable added a commit to highlyavailable/opentelemetry-go that referenced this issue May 30, 2025
@highlyavailable
prometheus: validate exponential histogram scale range (open-telemetr…
b0b9ac2 
…y#6779)
highlyavailable added a commit to highlyavailable/opentelemetry-go that referenced this issue May 30, 2025
@highlyavailable
prometheus: validate exponential histogram scale range for Prometheus…
95c4664 
… compatibility

Implements scale validation and bucket re-aggregation for exponential histograms to ensure compatibility with Prometheus native histograms, which support scales in the range [-4, 8].

  - Reject scales below -4 with error logging (skip data point)
  - Downscale histograms with scale > 8 by re-aggregating buckets
  - Add downscaleExponentialBucket function that merges buckets when downscaling
  - Fix overflow/underflow checks to handle negative bucket offsets correctly
  - Add comprehensive tests for scale validation and bucket downscaling

Fixes open-telemetry#6779
highlyavailable added a commit to highlyavailable/opentelemetry-go that referenced this issue May 30, 2025
@highlyavailable
prometheus: validate exponential histogram scale range (open-telemetr…
941484e 
…y#6779)
highlyavailable added a commit to highlyavailable/opentelemetry-go that referenced this issue May 30, 2025
@highlyavailable
prometheus: validate exponential histogram scale range for Prometheus…
324b4ec 
… compatibility

Implements scale validation and bucket re-aggregation for exponential histograms to ensure compatibility with Prometheus native histograms, which support scales in the range [-4, 8].

  - Reject scales below -4 with error logging (skip data point)
  - Downscale histograms with scale > 8 by re-aggregating buckets
  - Add downscaleExponentialBucket function that merges buckets when downscaling
  - Fix overflow/underflow checks to handle negative bucket offsets correctly
  - Add comprehensive tests for scale validation and bucket downscaling

Fixes open-telemetry#6779
highlyavailable added a commit to highlyavailable/opentelemetry-go that referenced this issue May 30, 2025
@highlyavailable
feat(prometheus): validate exponential histogram scale range
764b9f2 
Fixes open-telemetry#6779

Add scale validation for Prometheus exponential histograms to ensure
compatibility with Prometheus native histogram format.

Changes:
- Validate scale is within Prometheus supported range [-4, 8]
- Reject scales below -4 (log error and skip data point)
- Downscale histograms with scale > 8 by re-aggregating buckets
- Add comprehensive test coverage for scale validation and downscaling
- Implement downscaling logic based on OpenTelemetry Collector Contrib

The downscaling implementation merges buckets using bit-shifting to
maintain accuracy while conforming to Prometheus limitations.
@pellared pellared modified the milestones: v1.37.0, Subsequent v1.37.0 Jun 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@highlyavailable highlyavailable

Labels
good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Milestone
v1.38.0
4 participants
@dashpole @pellared @MrAlias @highlyavailable