Fix metadata propagation from scrape to otel components (#4098)

* Fix metadata propagation from scrape to otel components

* Use test context

* PR review polishing

* Docs fixes

Author: ptodev

CHANGELOG.md

@@ -55,6 +55,9 @@ Main (unreleased)
 
 - Add otel collector converter for `otelcol.receiver.googlecloudpubsub`. (@kalleep)
 
+- (_Experimental_) Add a `honor_metadata` configuration argument to the `prometheus.scrape` component.
+  When set to `true`, it will propagate metric metadata to downstream components.
+  
 ### Enhancements
 
 - Fix `pyroscope.write` component's `AppendIngest` method to respect configured timeout and implement retry logic. The method now properly uses the configured `remote_timeout`, includes retry logic with exponential backoff, and tracks metrics for sent/dropped bytes and profiles consistently with the `Append` method. (@korniltsev)

docs/sources/reference/components/loki/loki.write.md

@@ -127,11 +127,7 @@ When `retry_on_http_429` is enabled, the retry mechanism is governed by the back
 
 ### `queue_config`
 
-> **EXPERIMENTAL**: This is an [experimental][] feature.
-> Experimental features are subject to frequent breaking changes, and may be removed with no equivalent replacement.
-> The `stability.level` flag must be set to `experimental` to use the feature.
-
-[experimental]: https://grafana.com/docs/release-life-cycle/
+{{< docs/shared lookup="stability/experimental_feature.md" source="alloy" version="<ALLOY_VERSION>" >}}
 
 The optional `queue_config` block configures, when WAL is enabled, how the underlying client queues batches of logs sent to Loki.
 Refer to [Write-Ahead block](#wal) for more information.
@@ -149,9 +145,7 @@ The following arguments are supported:
 
 ### `wal`
 
-> **EXPERIMENTAL**: This is an [experimental][] feature.
-> Experimental features are subject to frequent breaking changes, and may be removed with no equivalent replacement.
-> The `stability.level` flag must be set to `experimental` to use the feature.
+{{< docs/shared lookup="stability/experimental_feature.md" source="alloy" version="<ALLOY_VERSION>" >}}
 
 The optional `wal` block configures the Write-Ahead Log (WAL) used in the Loki remote-write client.
 To enable the WAL, you must include the `wal` block in your configuration.

docs/sources/reference/components/otelcol/otelcol.exporter.loadbalancing.md

@@ -86,9 +86,10 @@ where the list of resolved endpoints changes frequently due to deployments and s
 
 > **EXPERIMENTAL**: Metrics support in `otelcol.exporter.loadbalancing` is an [experimental][] feature.
 > Experimental features are subject to frequent breaking changes, and may be removed with no equivalent replacement.
-> The `stability.level` flag must be set to `experimental` to use the feature.
+> To enable and use an experimental feature, you must set the `stability.level` [flag][] to `experimental`.
 
 [experimental]: https://grafana.com/docs/release-life-cycle/
+[flag]: https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/cli/run/
 
 ## Blocks
 

docs/sources/reference/components/prometheus/prometheus.scrape.md

@@ -53,6 +53,7 @@ You can use the following arguments with `prometheus.scrape`:
 | `http_headers`                       | `map(list(secret))`     | Custom HTTP headers to be sent along with each request. The map key is the header name.                                  |                                                                                                  | no       |
 | `honor_labels`                       | `bool`                  | Indicator whether the scraped metrics should remain unmodified.                                                          | `false`                                                                                          | no       |
 | `honor_timestamps`                   | `bool`                  | Indicator whether the scraped timestamps should be respected.                                                            | `true`                                                                                           | no       |
+| `honor_metadata`                     | `bool`                  | (Experimental) Indicator whether metric metadata should be sent to downstream components.                                | `false`                                                                                          | no       |
 | `job_name`                           | `string`                | The value to use for the job label if not already set.                                                                   | component name                                                                                   | no       |
 | `label_limit`                        | `uint`                  | More than this many labels post metric-relabeling causes the scrape to fail.                                             |                                                                                                  | no       |
 | `label_name_length_limit`            | `uint`                  | More than this label name length post metric-relabeling causes the scrape to fail.                                       |                                                                                                  | no       |
@@ -79,6 +80,17 @@ You can use the following arguments with `prometheus.scrape`:
 | `target_limit`                       | `uint`                  | More than this many targets after the target relabeling causes the scrapes to fail.                                      |                                                                                                  | no       |
 | `track_timestamps_staleness`         | `bool`                  | Indicator whether to track the staleness of the scraped timestamps.                                                      | `false`                                                                                          | no       |
 
+> **EXPERIMENTAL**: The `honor_metadata` argument is an [experimental][] feature.
+> Enabling it may increase resource consumption, particularly if a lot of metrics with different names are ingested.
+> Not all downstream components may be compatible with Prometheus metadata yet.
+> For example, `otelcol.receiver.prometheus` may work, but `prometheus.remote_write` may not.
+> Support for more components will be added soon.
+> Experimental features are subject to frequent breaking changes, and may be removed with no equivalent replacement.
+> To enable and use an experimental feature, you must set the `stability.level` [flag][] to `experimental`.
+
+[experimental]: https://grafana.com/docs/release-life-cycle/
+[flag]: https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/cli/run/
+
 At most, one of the following can be provided:
 
 * [`authorization`][authorization] block

docs/sources/reference/stdlib/array.md

@@ -81,7 +81,6 @@ Examples using discovery and exporter components:
 You can find more examples in the [tests][].
 
 [tests]: https://github.com/grafana/alloy/blob/main/syntax/vm/vm_stdlib_test.go
-[experimental]: https://grafana.com/docs/release-life-cycle/
 
 ## array.group_by
 

docs/sources/troubleshoot/debug.md

@@ -43,10 +43,7 @@ Click the {{< param "PRODUCT_NAME" >}} logo to navigate back to the home page.
 
 ### Graph page
 
-> **EXPERIMENTAL**: This is an [experimental][] feature.
-> Experimental features are subject to frequent breaking changes, and may be removed with no equivalent replacement.
-
-[experimental]: https://grafana.com/docs/release-life-cycle/
+{{< docs/shared lookup="stability/experimental_feature.md" source="alloy" version="<ALLOY_VERSION>" >}}
 
 {{< figure src="/media/docs/alloy/ui_graph_page_new_2.png" alt="Alloy UI graph page" >}}
 

internal/component/otelcol/exporter/prometheus/prometheus.go

@@ -87,7 +87,7 @@ func New(o component.Options, c Arguments) (*Component, error) {
 		return nil, err
 	}
 	ls := service.(labelstore.LabelStore)
-	fanout := prometheus.NewFanout(nil, o.ID, o.Registerer, ls)
+	fanout := prometheus.NewFanout(nil, o.ID, o.Registerer, ls, prometheus.NoopMetadataStore{})
 
 	converter := convert.New(o.Logger, fanout, convertArgumentsToConvertOptions(c))
 

internal/component/otelcol/receiver/prometheus/internal/transaction.go

@@ -542,8 +542,9 @@ func (t *transaction) Rollback() error {
 	return nil
 }
 
-func (t *transaction) UpdateMetadata(_ storage.SeriesRef, _ labels.Labels, _ metadata.Metadata) (storage.SeriesRef, error) {
-	// TODO: implement this func
+func (t *transaction) UpdateMetadata(_ storage.SeriesRef, lbls labels.Labels, m metadata.Metadata) (storage.SeriesRef, error) {
+	// There's no need to implement this.
+	// The metadata will be retrieved from the context.
 	return 0, nil
 }