Add ARC support

Author: Raghuadmn

.chloggen/arc-feature.yaml

@@ -0,0 +1,32 @@
+change_type: enhancement
+component: pkg/exporterhelper
+note: >
+  Add Adaptive Request Concurrency (ARC) to exporterhelper: a controller that
+  dynamically adjusts exporter concurrency based on observed RTT and
+  backpressure signals. Integrates with the queue/batch send path and exposes
+  new telemetry. Disabled by default; no behavior change unless enabled.
+issues: [14080]
+subtext: |
+  Configuration (exporterhelper / queue-batch settings):
+    - enabled: turn ARC on/off (default: false)
+    - initial_concurrency: starting permits (default: 1)
+    - max_concurrency: upper bound (default: 200)
+    - decrease_ratio: multiplicative decrease factor on pressure (default: 0.9)
+    - ewma_alpha: smoothing for RTT EWMA (default: 0.4)
+    - rtt_deviation_scale: N·σ spike threshold for decrease (default: 2.5)
+
+  Telemetry (new metrics):
+    - otelcol_exporter_arc_acquire_wait_ms (histogram; attrs: exporter, data_type)
+    - otelcol_exporter_arc_rtt_ms (histogram; attrs: exporter, data_type)
+    - otelcol_exporter_arc_failures (sum, monotonic; attrs: exporter, data_type)
+    - otelcol_exporter_arc_backoff_events (sum, monotonic; attrs: exporter, data_type)
+    - otelcol_exporter_arc_limit_changes (sum, monotonic; attrs: exporter, data_type, direction=up|down)
+    - otelcol_exporter_arc_limit (async gauge; attrs: exporter, data_type)
+    - otelcol_exporter_arc_permits_in_use (async gauge; attrs: exporter, data_type)
+
+  Notes:
+    - ARC increases concurrency additively and decreases multiplicatively on
+      backpressure or RTT spikes (mean + rtt_deviation_scale·σ).
+    - Direction attribute values: "up", "down".
+change_logs: [user]
+

exporter/exporterhelper/README.md

@@ -29,6 +29,7 @@ The following configuration options can be modified:
     - `bytes`: the size of serialized data in bytes (the least performant option).
   - `queue_size` (default = 1000): Maximum size the queue can accept. Measured in units defined by `sizer`
   - `batch`: see below.
+  - `arc`: see below.
 
 #### Sending queue batch settings
 
@@ -50,6 +51,19 @@ Available `batch::sizer` options:
 
 - `items`: number of the smallest parts of each signal (spans, metric data points, log records);
 - `bytes`: the size of serialized data in bytes (the least performant option).
+
+#### Sending queue Adaptive Concurrency Limiter (ARC) Settings
+
+The Adaptive Concurrency Limiter (ARC) dynamically adjusts the number of concurrent requests (`num_consumers`) based on observed RTTs and backpressure signals. It aims to maximize throughput while minimizing errors and latency. It is disabled by default.
+
+- `arc`
+  - `enabled` (default = false): Set to `true` to enable ARC.
+  - `initial_limit` (default = 1): The starting concurrency limit.
+  - `max_concurrency` (default = 200): The maximum number of concurrent requests ARC will allow.
+  - `decrease_ratio` (default = 0.9): The multiplicative factor to apply when decreasing the limit (e.g., 0.9 = 10% decrease).
+  - `ewma_alpha` (default = 0.4): The smoothing factor for the EWMA (Exponentially Weighted Moving Average) of RTTs.
+  - `deviation_scale` (default = 2.5): The number of standard deviations from the mean RTT to tolerate before triggering a backoff.
+
 ### Timeout
 
 - `timeout` (default = 5s): Time to wait per individual attempt to send data to a backend

exporter/exporterhelper/documentation.md

@@ -6,6 +6,68 @@
 
 The following telemetry is emitted by this component.
 
+### otelcol_exporter_arc_acquire_wait_ms
+
+Time a worker waited to acquire an ARC permit. [Alpha]
+
+| Unit | Metric Type | Value Type | Stability |
+| ---- | ----------- | ---------- | --------- |
+| ms | Histogram | Int | Alpha |
+
+### otelcol_exporter_arc_backoff_events
+
+Number of ARC backoff (shrink) events triggered by error or RTT signal. [Alpha]
+
+| Unit | Metric Type | Value Type | Monotonic | Stability |
+| ---- | ----------- | ---------- | --------- | --------- |
+| {events} | Sum | Int | true | Alpha |
+
+### otelcol_exporter_arc_failures
+
+Number of requests considered failures by ARC (feeds adaptive shrink). [Alpha]
+
+| Unit | Metric Type | Value Type | Monotonic | Stability |
+| ---- | ----------- | ---------- | --------- | --------- |
+| {requests} | Sum | Int | true | Alpha |
+
+### otelcol_exporter_arc_limit
+
+Current ARC dynamic concurrency limit. [Alpha]
+
+| Unit | Metric Type | Value Type | Stability |
+| ---- | ----------- | ---------- | --------- |
+| {permits} | Gauge | Int | Alpha |
+
+### otelcol_exporter_arc_limit_changes
+
+Number of times ARC changed its concurrency limit. [Alpha]
+
+| Unit | Metric Type | Value Type | Monotonic | Stability |
+| ---- | ----------- | ---------- | --------- | --------- |
+| {events} | Sum | Int | true | Alpha |
+
+#### Attributes
+
+| Name | Description | Values |
+| ---- | ----------- | ------ |
+| direction | up or down | Str: ``up``, ``down`` |
+
+### otelcol_exporter_arc_permits_in_use
+
+Number of permits currently acquired. [Alpha]
+
+| Unit | Metric Type | Value Type | Stability |
+| ---- | ----------- | ---------- | --------- |
+| {permits} | Gauge | Int | Alpha |
+
+### otelcol_exporter_arc_rtt_ms
+
+Request round-trip-time measured by ARC (from permit acquire to release). [Alpha]
+
+| Unit | Metric Type | Value Type | Stability |
+| ---- | ----------- | ---------- | --------- |
+| ms | Histogram | Int | Alpha |
+
 ### otelcol_exporter_enqueue_failed_log_records
 
 Number of log records failed to be added to the sending queue. [Alpha]

exporter/exporterhelper/generated_package_test.go

@@ -16,8 +16,8 @@ require (
 	go.opentelemetry.io/collector/consumer/consumererror v0.139.0
 	go.opentelemetry.io/collector/consumer/consumertest v0.139.0
 	go.opentelemetry.io/collector/exporter v1.45.0
-	go.opentelemetry.io/collector/exporter/exportertest v0.139.0
-	go.opentelemetry.io/collector/extension/extensiontest v0.139.0
+	go.opentelemetry.io/collector/exporter/exportertest v0.138.0
+	go.opentelemetry.io/collector/extension/extensiontest v0.138.0
 	go.opentelemetry.io/collector/extension/xextension v0.139.0
 	go.opentelemetry.io/collector/featuregate v1.45.0
 	go.opentelemetry.io/collector/pdata v1.45.0
@@ -33,6 +33,7 @@ require (
 	go.uber.org/goleak v1.3.0
 	go.uber.org/multierr v1.11.0
 	go.uber.org/zap v1.27.0
+	google.golang.org/grpc v1.76.0
 	google.golang.org/protobuf v1.36.10
 )
 
@@ -61,9 +62,9 @@ require (
 	go.opentelemetry.io/collector/receiver/receivertest v0.139.0 // indirect
 	go.opentelemetry.io/collector/receiver/xreceiver v0.139.0 // indirect
 	go.yaml.in/yaml/v3 v3.0.4 // indirect
-	golang.org/x/sys v0.35.0 // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20250804133106-a7a43d27e69b // indirect
-	google.golang.org/grpc v1.76.0 // indirect
+	golang.org/x/sys v0.36.0 // indirect
+	golang.org/x/text v0.29.0 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20250825161204-c5933d9347a5 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )