[BUG] Synthetic Source Compatibility Error in Elastic.Serilog.Sinks 9.x

[amyboose]

ECS integration/library project(s) (e.g. Elastic.CommonSchema.Serilog): Elastic.Serilog.Sinks (Elastic.CommonSchema.Elasticsearch)

ECS schema version: 9.0.0

ECS .NET assembly version: 9.0.0

Elasticsearch version: 8.13.2

.NET framework / OS: .NET Core 9 / Windows Server 2020

Description of the problem, including expected versus actual behavior:
After upgrading to Elastic.Serilog.Sinks 9.x, the library attempts to create component templates with the synthetic_source_keep parameter, which is not supported in some Elasticsearch 8.13.2 configurations, despite the version number suggesting compatibility.

Error Message:

Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.Exception: Failure to create component template `ecs_9.0.0_base` for logs-srv-*: Invalid Elasticsearch response built from a unsuccessful (400) low level call on PUT: /_component_template/ecs_9.0.0_base
 Exception: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""

# Audit trail of this API call:
 - [1] BadResponse: Node: http://localhost:9200/ Took: 00:00:00.0190869
# OriginalException: Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""
# Request:
{
  "_meta": {
    "documentation": "https://www.elastic.co/guide/en/ecs/current/ecs-base.html",
    "ecs_version": "9.0.0"
  },
  "template": {
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "labels": {
          "type": "object"
        },
        "message": {
          "type": "match_only_text"
        },
        "tags": {
          "ignore_above": 1024,
          "synthetic_source_keep": "none",
          "type": "keyword"
        }
      }
    }
  }
}

# Response:
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]","caused_by":{"type":"mapper_parsing_exception","reason":"unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]"}},"status":400}

 ---> Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""
   --- End of inner exception stack trace ---
   at Elastic.Ingest.Elasticsearch.ElasticsearchChannelBase`2.PutComponentTemplate(BootstrapMethod bootstrapMethod, String name, String body)
   at Elastic.Ingest.Elasticsearch.CommonSchema.EcsDataStreamChannel`1.BootstrapElasticsearch(BootstrapMethod bootstrapMethod, String ilmPolicy)
   at Elastic.Serilog.Sinks.ElasticsearchSink`1..ctor(ElasticsearchSinkOptions`1 options)
   at Elastic.Serilog.Sinks.ElasticsearchSink..ctor(ElasticsearchSinkOptions options)
   at Elastic.Serilog.Sinks.ConfigSinkExtensions.Elasticsearch(LoggerSinkConfiguration loggerConfiguration, BootstrapMethod bootstrapMethod, ICollection`1 nodes, Boolean useSniffing, String dataStream, String ilmPolicy, String apiKey, String username, String password, Nullable`1 includeHost, Nullable`1 includeActivity, Nullable`1 includeProcess, Nullable`1 includeUser, ICollection`1 filterProperties, Nullable`1 maxRetries, Nullable`1 maxConcurrency, Nullable`1 maxInflight, Nullable`1 maxExportSize, Nullable`1 maxLifeTime, Nullable`1 fullMode, Uri proxy, String proxyUsername, String proxyPassword, String fingerprint, Boolean debugMode, LoggingLevelSwitch levelSwitch, LogEventLevel restrictedToMinimumLevel)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   --- End of inner exception stack trace ---
   at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at Serilog.Settings.Configuration.ConfigurationReader.CallConfigurationMethods(ILookup`2 methods, IReadOnlyCollection`1 configurationMethods, Object receiver)
   at Serilog.Settings.Configuration.ConfigurationReader.ApplySinks(LoggerConfiguration loggerConfiguration)
   at Serilog.Settings.Configuration.ConfigurationReader.Configure(LoggerConfiguration loggerConfiguration)
   at Serilog.Configuration.LoggerSettingsConfiguration.Settings(ILoggerSettings settings)
   at Serilog.ConfigurationLoggerConfigurationExtensions.Configuration(LoggerSettingsConfiguration settingConfiguration, IConfiguration configuration, ConfigurationReaderOptions readerOptions)
   at Program.<Main>$(String[] args)
   at Program.<Main>(String[] args)

Steps to reproduce:

1. Have an Elasticsearch 8.13.2 cluster (free version)
2. Install Elastic.Serilog.Sinks 9.0.0

[thomas-girotto]

I'm having exactly the same issue, any news or workaround ?