Client retries on request failure

[TimQuist]

Description

Hi,

I am wondering if there are any plans for the new client to support the retrying of requests on failure?

I noticed when searching for this that some of the other clients seem to have this functionality:

• .net
• php
• ruby

For the previous RestHighLevelClient I have built my own retry mechanism and I am wondering if I should do the same for the new client?

One of the 'issues' I have run into when looking into implementing this mechanism myself was the fact that the client only throws a generic ElasticsearchException with ErrorResponse. Making the error handling a bit annoying to work with since most Java retry libraries depend on usage of exceptions types to determine if you want to conditionally retry a request or not.

An option would be to re-create and throw exceptions based on the ErrorResponse for the retry-able errors, or just implement the retry system myself completely without a library. But I first want to check in here before I go down that route.

I'd be willing to contribute, even though I'd have to get familiar with the codebase first.

Thanks in advance!

[Enerccio]

I would wrap Low Level Client in an retriable adapter, using spring-retry.

[TimQuist]

I would wrap Low Level Client in an retriable adapter, using spring-retry.

Thanks for the suggestion, I see the Low Level Client indeed has more in place for retrying.

[nithril]

It looks the rest client is still using the httpclient 4 that is not supporting retry for the async client :/

[swallez]

@TimQuist you will get an ElasticsearchException only if the http request was successful at the http level and Elasticsearch returned an error. All other network-level errors are reported as IOException or one of its subclasses, and you should be able check their actual type.

The Low Level Client also does retry when configured when several nodes: on network failure, it will retry up to the number of nodes.

@nithril the Low Level Client has stayed on httpclient 4 for backwards compatibility reasons. We're considering writing a new implementation of transport based on httpclient 5 that would also allow removing the dependency to the Low Level Client. It's not there yet though.

[smandyaj]

@swallez i had a quick question : What is the case when a single document fails to deserialize for an operation like
searchRs = esClient.search(searchRq, IKnowTheObject.class);
when it tries to fetch max(10000) documents ?

And
if (ex.getCause() instanceof JsonpMappingException || ex.getCause() instanceof JsonParsingException)
is this correct way to check for such exceptions ?

[l-trotta]

I'm closing this since we're tracking the effort to update the transport layer somewhere else, if anyone have further questions please consider writing on our discuss forum :)