[bug] Enrich methods can be called multiple times.

[TimothyMothra]

Component

OpenTelemetry.Instrumentation.Http

Package Version

Package Name	Version
OpenTelemetry.Instrumentation.AspNetCore	1.9.0
OpenTelemetry.Instrumentation.Http	1.9.0

Runtime Version

net8.0

Description

I've verified this behavior with the Http and AspNetCore Instrumentation libraries, but I suspect it applies to all Instrumentation libraries. For the sake of conversation, I'm going to focus on the HttpClient Instrumentation.

When configuring OpenTelemetry Instrumentation Libraries, if the AddHttpClientInstrumentation() method is called multiple times, any configured method delegates will be called multiple times.

This can be a problem when users consume a Distro package, such as Azure.Monitor.OpenTelemetry.AspNetCore.
In this example, the Distro internally adds Instrumentation Libraries.
If a user wants to add their configuration to these instrumentations, they may call AddHttpClientInstrumentation() an additional time to set their options.

Workaround

For the sake of conversation, I'm going to focus on the HttpClient Instrumentation.

As a workaround, users should use the Configure api builder.Services.Configure<HttpClientTraceInstrumentationOptions>().

Steps to Reproduce

EXAMPLE 1: HttpClient Instrumentation

using var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddHttpClientInstrumentation()
    .AddHttpClientInstrumentation(options =>
    {
        options.EnrichWithHttpRequestMessage = (activity, request) =>
        {
            Console.WriteLine($"****** EnrichWithHttpRequestMessage: {activity.Id} {request.RequestUri}");
        };
    })
    .AddConsoleExporter()
    .Build();

var client = new HttpClient();
var googleResponse = await client.GetStringAsync("https://www.google.com");

OUTPUT

****** EnrichWithHttpRequestMessage: 00-c8b9d61d40f12e8255c0eec002927ded-70aed8f7023b94fb-01 https://www.google.com/
****** EnrichWithHttpRequestMessage: 00-c8b9d61d40f12e8255c0eec002927ded-70aed8f7023b94fb-01 https://www.google.com/
Activity.TraceId:            c8b9d61d40f12e8255c0eec002927ded
Activity.SpanId:             70aed8f7023b94fb
Activity.TraceFlags:         Recorded
Activity.ActivitySourceName: System.Net.Http
Activity.DisplayName:        GET
Activity.Kind:               Client
Activity.StartTime:          2024-10-24T23:28:46.0328673Z
Activity.Duration:           00:00:00.2768286
Activity.Tags:
    http.request.method: GET
    server.address: www.google.com
    server.port: 443
    url.full: https://www.google.com/
    network.protocol.version: 1.1
    http.response.status_code: 200
Resource associated with Activity:
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.9.0
    service.name: unknown_service:ConsoleApp1

EXAMPLE 2: AspNetCore Instrumentation

var builder = WebApplication.CreateBuilder(args);

// Configure OpenTelemetry
builder.Services.AddOpenTelemetry().WithTracing(tracerProviderBuilder =>
{
    tracerProviderBuilder
        .AddAspNetCoreInstrumentation()
        .AddAspNetCoreInstrumentation(options =>
        {
            options.EnrichWithHttpRequest = (activity, request) =>
            {
                Console.WriteLine($"****** EnrichWithHttpRequest: {activity.Id} {request.Path}");
            };
        })
        .AddConsoleExporter();
});

var app = builder.Build();

app.MapGet("/", () => "Hello, World!");

var hostTask = app.RunAsync();

// Create an HttpClient to make a request to the in-process web app
var client = new HttpClient();
var response = await client.GetStringAsync("http://localhost:5000");
Console.WriteLine(response);

await hostTask;

OUTPUT

****** EnrichWithHttpRequest: 00-fe638d554831f3f30dc25eeca2cba265-fadc3e15c6641a5b-01 /
****** EnrichWithHttpRequest: 00-fe638d554831f3f30dc25eeca2cba265-fadc3e15c6641a5b-01 /
Activity.TraceId:            fe638d554831f3f30dc25eeca2cba265
Activity.SpanId:             fadc3e15c6641a5b
Activity.TraceFlags:         Recorded
Activity.ActivitySourceName: Microsoft.AspNetCore
Activity.DisplayName:        GET /
Activity.Kind:               Server
Activity.StartTime:          2024-10-24T23:30:17.5675625Z
Activity.Duration:           00:00:00.0728921
Activity.Tags:
    server.address: localhost
    server.port: 5000
    http.request.method: GET
    url.scheme: http
    url.path: /
    network.protocol.version: 1.1
    http.route: /
    http.response.status_code: 200
Resource associated with Activity:
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.9.0
    service.name: unknown_service:ConsoleApp1

EXAMPLE 3: AzureMonitor Distro

Internally, UseAzureMonitor() already calls AddAspNetCoreInstrumentation.
Here a user may call this an additional time to set their own customizations.

builder.Services.AddOpenTelemetry().UseAzureMonitor()
    .WithTracing(builder =>
    {
        builder.AddAspNetCoreInstrumentation(options =>
        {
            options.EnrichWithHttpRequest = (activity, HttpResponse) =>
            {
                Console.WriteLine($"****** EnrichWithHttpRequest: {activity.Id} {request.Path}");
            };
        });
    });

Expected Result

configured delegates would only be called once.

Actual Result

configured delegates are called however many times that Instrumentation library was added.

Additional Context

No response