docs: add sample clip training script with datasets & mlfow implement…

[aardoiz]

Hi! I was doing some experiments with CLIP implementation of SBERT and though that my script could be an improvement over the sample we have in the repository.

Pull Request overview

• Add a sample file to train CLIP using flickr8k from HF dataset and logging the process into MLFlow

aardoiz

[tomaarsen]

Hello!

Thanks for opening! There's indeed a big lack of training examples for CLIP. I'll try to run and review this after the weekend, but I like the idea of it a lot. I also haven't used mlflow myself yet, so that'll be interesting!

transformers, on which Sentence Transformers is built, has a direct integration for its training: https://mlflow.org/docs/latest/ml/deep-learning/transformers/index.html
So we might even be able to simplify the mlflow.start_run etc. code.

• Tom Aarsen

[tomaarsen]

I did some tests re:

transformers, on which Sentence Transformers is built, has a direct integration for its training: https://mlflow.org/docs/latest/ml/deep-learning/transformers/index.html
So we might even be able to simplify the mlflow.start_run etc. code.

And it turns out that the manual mlflow definition cannibalizes the automatic integration with transformers, so I think we're best off removing it. I also updated the preprocessing and loss function a bit based on my experiences with Sentence Transformers training.

Here's a screenshot of the new MLFlow outputs:

What do you think? I think it might be ready to merge.

• Tom Aarsen