Getting embeddings

[johnyquest7]

thank you for creatng this awesome repository.
How do I get embeddings out of the SwAV model for one image?

[KeremTurgutlu]

Assuming you trained with a learner such as:

dls = get_dls(resize, bs)
model = create_swav_model(arch=xresnet34, n_in=3, pretrained=False)
learn = Learner(dls, model, SWAVLoss(),
                cbs=[SWAV(crop_sizes=[size,96], 
                          num_crops=[2,6],
                          min_scales=[0.25,0.2],
                          max_scales=[1.0,0.35]),
                     TerminateOnNaNCallback()])

You can simply use learn.model.encoder and feed input image to extract it's embedding. Don't forget to process the image with the same transforms you used during training, like normalization.

If this doesn't help, please feel free to share link to a notebook that you have so that we can debug it together.

[johnyquest7]

Awesome

[KeremTurgutlu]

Until the official release of new version you can checkout this example notebook.

[johnyquest7]

Thank you for the detailed example. Will have to try it out.
It was very easy to train a model with your code. I am seeing pretty good results and it seems to be really fast!
Saw your to-do list - waiting for - Medical domain experiments/examples

[KeremTurgutlu]

It was very easy to train a model with your code. I am seeing pretty good results and it seems to be really fast!

This comment makes me very happy :) The new version I am working on is even faster and better, you can checkout few experiments here on imagewang benchmark

Saw your to-do list - waiting for - Medical domain experiments/examples

My plan is to start with chest X-rays and try it out on few downstream tasks such as Kaggle competitions or other leaderboards. It will be interesting and hard to find out which augmentations are going to work for medical data. Let me know if you already have an idea :)

[johnyquest7]

CLAHE worked better for one of my medical projects. Not sure whether it was a random finding. https://albumentations.ai/docs/api_reference/augmentations/transforms/#albumentations.augmentations.transforms.CLAHE