Bump sentence-transformers from 2.2.2 to 2.7.0 #1038

dependabot · 2024-12-23T14:03:27Z

Bumps sentence-transformers from 2.2.2 to 2.7.0.

Release notes

Sourced from sentence-transformers's releases.

v2.7.0 - CachedGISTEmbedLoss, easy Matryoshka inference & evaluation, CrossEncoder, Intel Gaudi2

This release introduces a new promising loss function, easier inference for Matryoshka models, new functionality for CrossEncoders and Inference on Intel Gaudi2, along much more.

Install this version with
pip install sentence-transformers==2.7.0
New loss function: CachedGISTEmbedLoss (#2592)

For a number of years, MultipleNegativesRankingLoss (also known as SimCSE, InfoNCE, in-batch negatives loss) has been the state of the art in embedding model training. Notably, this loss function performs better with a larger batch size.

Recently, various improvements have been introduced:

CachedMultipleNegativesRankingLoss was introduced, which allows you to pick much higher batch sizes (e.g. 65536) with constant memory.

GISTEmbedLoss takes a guide model to guide the in-batch negative sample selection. This prevents false negatives, resulting in a stronger training signal.

Now, @JacksonCakes has combined these two approaches to produce the best of both worlds: CachedGISTEmbedLoss. This loss function allows for high batch sizes with constant memory usage, while also using a guide model to assist with the in-batch negative sample selection.

As can be seen in our Loss Overview, this model should be used with (anchor, positive) pairs or (anchor, positive, negative) triplets, much like MultipleNegativesRankingLoss, CachedMultipleNegativesRankingLoss, and GISTEmbedLoss. In short, any example using those loss functions can be updated to use CachedGISTEmbedLoss! Feel free to experiment, e.g. with this training script.

Automatic Matryoshka model truncation (#2573)

Sentence Transformers v2.4.0 introduced Matryoshka models: models whose embeddings are still useful after truncation. Since then, many useful Matryoshka models have been trained.

As of this release, the truncation for these Matryoshka embedding models can be done automatically via a new truncate_dim constructor argument:
from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim
matryoshka_dim = 64
model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True, truncate_dim=matryoshka_dim)
embeddings = model.encode(
[
"search_query: What is TSNE?",
"search_document: t-distributed stochastic neighbor embedding (t-SNE) is a statistical method for visualizing high-dimensional data by giving each datapoint a location in a two or three-dimensional map.",
"search_document: Amelia Mary Earhart was an American aviation pioneer and writer.",
]
)
print(embeddings.shape)
=> [3, 64]
similarities = cos_sim(embeddings[0], embeddings[1:])
=> tensor([[0.7839, 0.4933]])
Extra information:

Matryoshka Embeddings Documentation

Matryoshka Embeddings Blogpost

Model truncation in all evaluators (#2582)

... (truncated)

Commits

f558eae Release v2.7.0
43f4a22 Update CLIP tokenizer (#2599)
d50a04a Fix search bar on sbert.net (#2597)
38ab549 Add CachedGISTEmbedLoss (#2592)
efa3ed1 feat: add trust remote code to cross encoders (#2595)
57ea98c Update transformers requirement in setup.py to match requirements.txt (#2589)
f60a45b corrected comment from kmeans to agglomerative (#2590)
d3d767c [fix] Matryoshka training always patch original forward, and check matryosh...
d105ec8 Add ST annotation to evaluators (#2586)
99674c7 Add truncation support in evaluators (#2582)
Additional commits viewable in compare view

Most Recent Ignore Conditions Applied to This Pull Request

Dependency Name	Ignore Conditions
sentence-transformers	[>= 3.3.dev0, < 3.4]
sentence-transformers	[>= 3.dev0, < 4]

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [sentence-transformers](https://github.com/UKPLab/sentence-transformers) from 2.2.2 to 2.7.0. - [Release notes](https://github.com/UKPLab/sentence-transformers/releases) - [Commits](UKPLab/sentence-transformers@v2.2.2...v2.7.0) --- updated-dependencies: - dependency-name: sentence-transformers dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>