Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cmd/mdatagen] add field for configuring individual metric interval #12317

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
26 changes: 26 additions & 0 deletions .chloggen/configure-interval-on-metric-level.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. otlpreceiver)
component: cmd/mdatagen

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: add option to specify the scrape interval for individual metrics.

# One or more tracking issues or pull requests related to the change
issues: [12317]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: |
This is mainly useful for database related receivers. Metrics for databases are mainly retrieved via sql queries and some could be resource-intensive.

# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
13 changes: 13 additions & 0 deletions cmd/mdatagen/internal/loader_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,19 @@ func TestLoadMetadata(t *testing.T) {
},
Attributes: []AttributeName{"string_attr", "overridden_int_attr", "enum_attr", "slice_attr", "map_attr"},
},
"metric.with_interval": {
Enabled: true,
Interval: 15,
Description: "Monotonic cumulative sum int metric with string input_type enabled by default and contains a customized interval of 15 seconds.",
Unit: strPtr("s"),
Sum: &Sum{
MetricValueType: MetricValueType{pmetric.NumberDataPointValueTypeInt},
MetricInputType: MetricInputType{InputType: "string"},
AggregationTemporality: AggregationTemporality{Aggregation: pmetric.AggregationTemporalityCumulative},
Mono: Mono{Monotonic: true},
},
Attributes: []AttributeName{"string_attr", "overridden_int_attr", "enum_attr", "slice_attr", "map_attr"},
},
},
Telemetry: Telemetry{
Metrics: map[MetricName]Metric{
Expand Down
2 changes: 2 additions & 0 deletions cmd/mdatagen/internal/metric.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ type Metric struct {

// Attributes is the list of attributes that the metric emits.
Attributes []AttributeName `mapstructure:"attributes"`

Interval uint64 `mapstructure:"interval"`
}

type Stability struct {
Expand Down
18 changes: 18 additions & 0 deletions cmd/mdatagen/internal/samplereceiver/documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,24 @@ Monotonic cumulative sum int metric with string input_type enabled by default.
| slice_attr | Attribute with a slice value. | Any Slice |
| map_attr | Attribute with a map value. | Any Map |

### metric.with_interval

Monotonic cumulative sum int metric with string input_type enabled by default and contains a customized interval of 15 seconds.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| s | Sum | Int | Cumulative | true |

#### Attributes

| Name | Description | Values |
| ---- | ----------- | ------ |
| string_attr | Attribute with any string value. | Any Str |
| state | Integer attribute with overridden name. | Any Int |
| enum_attr | Attribute with a known set of string values. | Str: ``red``, ``green``, ``blue`` |
| slice_attr | Attribute with a slice value. | Any Slice |
| map_attr | Attribute with a map value. | Any Map |

## Optional Metrics

The following metrics are not emitted by default. Each of them can be enabled by applying the following configuration:
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ all_set:
enabled: true
metric.input_type:
enabled: true
metric.with_interval:
enabled: true
interval: 15
optional.metric:
enabled: true
optional.metric.empty_unit:
Expand Down Expand Up @@ -36,6 +39,8 @@ none_set:
enabled: false
metric.input_type:
enabled: false
metric.with_interval:
enabled: false
optional.metric:
enabled: false
optional.metric.empty_unit:
Expand Down
11 changes: 11 additions & 0 deletions cmd/mdatagen/internal/samplereceiver/metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,17 @@ metrics:
aggregation_temporality: cumulative
attributes: [ string_attr, overridden_int_attr, enum_attr, slice_attr, map_attr ]

metric.with_interval:
enabled: true
description: Monotonic cumulative sum int metric with string input_type enabled by default and contains a customized interval of 15 seconds.
unit: s
interval: 15
sum:
value_type: int
input_type: string
monotonic: true
aggregation_temporality: cumulative
attributes: [ string_attr, overridden_int_attr, enum_attr, slice_attr, map_attr ]
telemetry:
metrics:
batch_size_trigger_send:
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 9 additions & 1 deletion cmd/mdatagen/internal/templates/config.go.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,12 @@ import (
// MetricConfig provides common config for a particular metric.
type MetricConfig struct {
Enabled bool `mapstructure:"enabled"`

{{- range $_, $metric := .Metrics }}
{{- if ne $metric.Interval 0 }}
Interval uint64 `mapstructure:"interval"`
Copy link
Member

@bogdandrutu bogdandrutu Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does an interval mean for a metric? Most of our scrapers scrape all metrics in one RPC/REST call... How would this be supported?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the confusion in the description. This is primarily intended for database scrapers. Since these metrics are queried directly from the database and some of them are resource-intensive, it would be beneficial to allow certain metrics to be scraped less frequently.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make the generation of this config optional then.

So, do you mean that in the SQL world, every metric is a sql request?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot speak for all databases, but let's take MSSQL and PostgreSQL as examples. In the contrib repository, these metrics are collected by sending SQL queries.

For reference:

PostgreSQL: client.go#L293

MSSQL: queries.go#L14

I will update the code to make this behavior optional

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, please take a look, thanks!

{{- break }}
{{- end }}
{{- end }}
enabledSetByUser bool
}

Expand Down Expand Up @@ -41,6 +46,9 @@ func DefaultMetricsConfig() MetricsConfig {
{{- range $name, $metric := .Metrics }}
{{ $name.Render }}: MetricConfig{
Enabled: {{ $metric.Enabled }},
{{- if ne $metric.Interval 0 }}
Interval: {{ $metric.Interval }},
{{- end }}
},
{{- end }}
}
Expand Down
16 changes: 12 additions & 4 deletions cmd/mdatagen/internal/templates/config_test.go.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,12 @@ func TestMetricsBuilderConfig(t *testing.T) {
name: "all_set",
want: MetricsBuilderConfig{
Metrics: MetricsConfig{
{{- range $name, $_ := .Metrics }}
{{ $name.Render }}: MetricConfig{Enabled: true},
{{- range $name, $metric := .Metrics }}
{{- if ne $metric.Interval 0 }}
{{ $name.Render }}: MetricConfig{Enabled: true, Interval: {{ $metric.Interval }} },
{{- else }}
{{ $name.Render }}: MetricConfig{Enabled: true},
{{- end }}
{{- end }}
},
{{- if .ResourceAttributes }}
Expand All @@ -43,8 +47,12 @@ func TestMetricsBuilderConfig(t *testing.T) {
name: "none_set",
want: MetricsBuilderConfig{
Metrics: MetricsConfig{
{{- range $name, $_ := .Metrics }}
{{ $name.Render }}: MetricConfig{Enabled: false},
{{- range $name, $metric := .Metrics }}
{{- if ne $metric.Interval 0 }}
{{ $name.Render }}: MetricConfig{Enabled: false, Interval: {{ $metric.Interval }} },
{{- else }}
{{ $name.Render }}: MetricConfig{Enabled: false},
{{- end }}
{{- end }}
},
{{- if .ResourceAttributes }}
Expand Down
5 changes: 4 additions & 1 deletion cmd/mdatagen/internal/templates/testdata/config.yaml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@ default:
all_set:
{{- if .Metrics }}
metrics:
{{- range $name, $_ := .Metrics }}
{{- range $name, $v := .Metrics }}
{{ $name }}:
enabled: true
{{- if $v.Interval }}
interval: {{ $v.Interval }}
{{- end }}
{{- end }}
{{- end }}
{{- if .ResourceAttributes }}
Expand Down
Loading