[Question] Pipelines execution in Elastic.Serilog.Sinks

[ManVit94]

Hello,
During migration from Serilog.Sinks.Elasticsearch to Elastic.Serilog.Sinks, I encountered missing functionality for configuring the ingest pipeline in ElasticsearchSinkOptions. The pipeline is used to set up preprocessors that select the index based on log field values and to drop some redundant logs. Is there any other way to specify the pipeline? I couldn’t find any similar issues except this unanswered one. If there is a way to specify the pipeline, please share it. It would be helpful to have the ability to specify which pipeline should be used.

Here is Serilog.Sinks.Elasticsearch example:

var serilogSinksElasticsearchOptions = new ElasticsearchSinkOptions(new Uri("{elasticUrl}"))
{
    PipelineName = "my-pipeline",
    CustomFormatter = new ElasticsearchJsonFormatter(),
    BatchAction = ElasticOpType.Create,
    ConnectionTimeout = TimeSpan.FromSeconds(30),
    ModifyConnectionSettings = connection => connection
        .BasicAuthentication("username", "password")
        .RequestTimeout(TimeSpan.FromSeconds(90))
        .MaximumRetries(3),
};

And corresponding Elastic.Serilog.Sinks:

loggerConfig.WriteTo.Elasticsearch([new Uri("{elsaticUrl}")],
    options =>
    {
        // options.PipelineName = "my-pipeline"; -- unfortunately no such option
        options.BootstrapMethod = BootstrapMethod.None;
        options.DataStream = new DataStreamName("my-data-stream");
        options.TextFormatting = new EcsTextFormatterConfiguration<LogEventEcsDocument>();
        options.ConfigureChannel = channelOptions =>
        {
            channelOptions.BufferOptions = new BufferOptions();
        };
        options.IlmPolicy = DataStreamName;
    },
    transport =>
    {
        transport.Authentication(new BasicAuthentication("username", "password"));
        transport.RequestTimeout(TimeSpan.FromSeconds(90));
        transport.MaximumRetries(3);
    });