feat: [ML] Support binary embeddings from Amazon Bedrock Titan (#125378) #126540
Conversation
|
💚 CLA has been signed |
elasticsearchmachine
added
v9.1.0
needs:triage
davidkyle
added
:ml
|
Pinging @elastic/ml-core (Team:ML) |
|
@elasticmachine test this please |
There was a problem hiding this comment.
Thanks for the contribution @demoncoder-crypto
We have a very similar PR open for adding embedding type to AWS Bedrock Cohere embedding models #126565
| @@ -11,8 +11,9 @@ public class AmazonBedrockConstants { | |||
| public static final String ACCESS_KEY_FIELD = "access_key"; | |||
| public static final String SECRET_KEY_FIELD = "secret_key"; | |||
| public static final String REGION_FIELD = "region"; | |||
| public static final String MODEL_FIELD = "model"; | |||
| public static final String MODEL_FIELD = "model_id"; | |||
There was a problem hiding this comment.
Changing the value of this field will break existing configurations.
| ) {} | ||
|
|
||
| protected AmazonBedrockServiceSettings(StreamInput in) throws IOException { | ||
| this.region = in.readString(); | ||
| this.model = in.readString(); | ||
| this.provider = in.readEnum(AmazonBedrockProvider.class); | ||
| this.rateLimitSettings = new RateLimitSettings(in); | ||
| if (in.getTransportVersion().onOrAfter(TransportVersions.V_9_0_0)) { // Version set for BWC |
There was a problem hiding this comment.
Please create a new TransportVersion in https://github.com/elastic/elasticsearch/blob/main/server/src/main/java/org/elasticsearch/TransportVersions.java#L220
See this for an example: https://github.com/elastic/elasticsearch/pull/126565/files#diff-85e782e9e33a0f8ca8e99b41c17f9d04e3a7981d435abf44a3aa5d954a47cd8f
We need 2 versions, one for this branch and another for the backport to 8.x
| if (embeddingType != DEFAULT_EMBEDDING_TYPE) { | ||
| builder.field(EMBEDDING_TYPE_FIELD, embeddingType.toString()); | ||
| } |
There was a problem hiding this comment.
| if (embeddingType != DEFAULT_EMBEDDING_TYPE) { | |
| builder.field(EMBEDDING_TYPE_FIELD, embeddingType.toString()); | |
| } | |
| builder.field(EMBEDDING_TYPE_FIELD, embeddingType.toString()); |
We prefer show the default values explicitly rather than hiding them.
| } | ||
|
|
||
| @Override | ||
| public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException { | ||
| builder.startObject(); | ||
| builder.field(INPUT_TEXT_FIELD, inputText); | ||
| if (embeddingType == AmazonBedrockEmbeddingType.BINARY) { |
There was a problem hiding this comment.
Always be explicit rather than depending on default values set in a 3rd party service. If Bedrock changes the default embedding then it will break any integrations that rely on the default value being float. Also of a new value is added to the AmazonBedrockEmbeddingType enum it won't be set in this request.
The G1 models do not support binary embeddings only the V2 models
https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-embed-text.html
When an user creates
|
@elasticmachine test this please |
[ML] Support binary embeddings from Amazon Bedrock Titan (#125378)
This change adds support for the
binaryembedding type offered by Amazon Titan text embedding models via the Amazon Bedrock connector in the Inference API.Changes:
embedding_typeparameter to the Bedrock service configuration (service_settings). It accepts"float"(default) or"binary"."embeddingType": "binary"when theembedding_typesetting is set tobinary.embedding_type. ReturnsTextEmbeddingFloatResultsorTextEmbeddingBytesResultsaccordingly.This allows users to leverage the binary embedding format provided by Amazon Titan models through the Elasticsearch Inference API.