T5Gemma2 Support

[contrebande-labs]

Hi! As per #3602, here is a WIP just to check if I'm on the right track. If you agree with the general idea, I will write some tests with the t5gemma2 weights. I don't expect you guys to work on the holidays so take your time and get back to me when you can.

-- Vincent

[tomaarsen]

Hello!

This is definitely in the right direction, matching what I would do as well. The only downside is that the T5GemmaConfig doesn't immediately expose the hidden_size, but instead uses a text_config which hosts the hidden_size. I've been working on adding this support in #3554:

https://github.com/tomaarsen/sentence-transformers/blob/3c55e8b14a1a5bbfb96da1be6938b92a6a4f5adf/sentence_transformers/base/models/Transformer.py#L874-L900

Compared to main:

sentence-transformers/sentence_transformers/models/Transformer.py

Lines 287 to 288 in c19c540

	def get_word_embedding_dimension(self) -> int:
	return self.auto_model.config.hidden_size

I've done some tests, and it looks like this PR works once the get_word_embedding_dimension-changes from #3554 are merged.

---

I made some small changes, but it won't let me push that into this PR sadly, as I don't have the required permissions on the PR host repository. You can see the commit here: 7b429c8

Guard the T5Gemma2 import for transformers <5 compat, reformat

• Tom Aarsen

[contrebande-labs]

I added your changes. I didn't enable commiter push on this PR because I wanted to keep track of things. If you have other changes, let me know and I will make these changes.

I would now like to add examples and tests before I take this PR out of draft mode. I would like to implement a full quantization-aware FR-EN biilingual matryoshka training and subsequent ONNX export and OpenVINO ONNX Runtime inference. What do you think?

[contrebande-labs]

I forgot get_word_embedding_dimension() now it's done. I left out the TIMM stuff, however.

[contrebande-labs]

Hi Tom!

With the latest commit, loading the model now works:

st_pytorch_model = SentenceTransformer("google/t5gemma-2-270m-270m", backend="torch", device="xpu", trust_remote_code=True, local_files_only=False, model_kwargs={"dtype": torch.float32}).eval()

The code is a mix of your other PRs with what's currently on the main branch (e.g. model -- > auto_model). But the loading works. Did you intend to merge your other PRs first? If so, I can rebase once done. Otherwise, let me know how you suggest I make tests and examples.

[contrebande-labs]

Hi Tom. Or I can just turn over this PR to you so you can pull it and I can make another PR for examples and tests.

[contrebande-labs]

Hi Tom!

What are your plans for this PR ? Thanks!

Vincent