Text_expansion from InferenceConfig is not supported upon creating a Pipeline

[adrianpbv]

Java API client version

8.9.1

Java version

8

Elasticsearch Version

8.9.1

Problem description

I'm trying to create a pipeline that ingest data with the ELSER model. In the class PutPipelineRequest, InferenceConfig doesn’t support “text_expansion” value from the json file, an error is thrown:

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.ingest.InferenceConfig: Unknown field 'text_expansion' (JSON path: processors[0].inference.inference_config.text_expansion)

It seems that InferenceConfig only supports regression and classification types. As the ELSER model was recently introduced this is missed from the JavaApiClient v8.9.1, that is supposed to be updated with features released in Elastic v8.9.1.

The code to create the pipeline programmatically in java is:

InputStream input = new ClassPathResource("elser_pipeline_config.json").getInputStream();

PutPipelineRequest request = PutPipelineRequest.of(pl->pl
                    .id("my_pipeline_test")
                    .withJson(input)
            );
elasticsearchClient.ingest().putPipeline(request);

// elser_pipeline_config.json :
{
  "description":"My test elser pipeline",
  "processors": [
    {
      "inference": {
        "model_id": ".elser_model_1",
        "target_field": "ml",
        "field_map": { 
          "text": "text_field"
        },
        "inference_config": {
          "text_expansion": { 
            "results_field": "tokens"
          }
        }
      }
    }
  ]
}

As a temporal solution I created this workaround if someone runs into the same error.

[wnm3]

I have a similar problem trying to use the intfloat__multilingual-e5-base which requires an inferenceConfig for text_embedding -- see this thread for details.

Thank you for the workaround.

[l-trotta]

Hello, thank you for reporting this. The definition of both the InferencePipelineAggregation and Inference config seems outdated, we will update it in the API specification used to produce the Java code. Once it's fixed the Java client code will be updated to solve this.