Support semantic reranking using contextual snippets instead of entire field text

[kderusso]

Followup to the POC described in #128255

Adds the ability to rerank based on a smaller number of snippets.

Example, with a default of one snippet:

GET test/_search
{
  "retriever": {
    "text_similarity_reranker": {
      "retriever": {
        "standard": {
          "query": {
            "term": {
              "other": "lotr"
            }
          }
        }
      },
      "rank_window_size": 2,
      "field": "semantic_text",
      "inference_text": "are all who wander lost?",
      "snippets": { }
    }
  }
}

Example, specifying snippets:

GET test/_search
{
  "retriever": {
    "text_similarity_reranker": {
      "retriever": {
        "standard": {
          "query": {
            "term": {
              "other": "lotr"
            }
          }
        }
      },
      "rank_window_size": 2,
      "field": "semantic_text",
      "inference_text": "are all who wander lost?",
      "snippets": {
        "num_snippets": 3
      }
    }
  }
}

Not specifying snippets will continue to send the entire field contents into the reranker model.

[jimczi]

I did a first pass and left some comments. I think we can better isolate this change in the text similarity rank builder.

[cla-checker-service]

❌ Author of the following commits did not sign a Contributor Agreement:
9ae38c9, 8a82b13, 7c6848d, b6aaf8f, 2418e41

Please, read and sign the above mentioned agreement if you want to contribute to this project

[kderusso]

cla/check

[jimczi]

Should we also set noMatchSize to ensure that we always get a snippet for every document?

[jimczi]

Do we really need the separation here between TextSimilarityRerankingRankFeaturePhaseRankShardContext and RerankingRankFeaturePhaseRankShardContext? I don't see RerankingRankFeaturePhaseRankShardContext being used elsewhere so maybe we can just have TextSimilarityRerankingRankFeaturePhaseRankShardContext and implement the full logic there?

[jimczi]

Is it a leftover?

[jimczi]

Is the separation between SnippetRankInput and RerankSnippetConfig necessary? Why not injecting numSnippets and snippetQueryBuilder directly here?

[jimczi]

This is currently used as a character limit and not as a token limit. I don't understand why you're separating the RERANK_TOKEN_SIZE_LIMIT and the DEFAULT_TOKEN_SIZE_LIMIT. Currently the issue is that HighlightBuilder#fragmentSize sets the size of the fragment in terms of number of characters and not tokens.
512 being the context length of the Elastic re-ranker, we can hack temporarily by multiplying with the average length of a token in English. 4096 seems high since we'd expect an average of 8 characters per token. The literature on the topic is more on an average of 4-5 chars per token even less if we consider the model's vocabulary and tokenisation (wordpiece, ...).