Preprocess for torch.Tensor instead of PIL Image?

[AnsonSavage]

Howdy!

open_clip is awesome! I'm just getting started experimenting with all these awesome models.

I'm using cosine similarity in CLIP space as a loss for a project I'm working on. One of the things I'm noticing is that the preprocess variables returned by the create_model_and_transforms function expect a PIL Image. Because I'm optimizing against similarity in CLIP space, I need the inputs to be tensors so that gradients can propagate.

Maybe I've missed it, but are there any utilities available in this repository for getting preprocess callables that operate on the tensors?

Thanks!
Anson

[AnsonSavage]

(In the meantime, I just vibe-coded up this converter function, but it's a lot of duplicate work just to sidestep those initial Image to Tensor conversions...)
clip_utils.py

[rwightman]

@AnsonSavage most of the transforms are already fine with torch.Tensor, it was just the forced calls to ToTensor and img.conver('RGB'), I think #1122 should fix that?

[AnsonSavage]

Hey, this is awesome! This should definitely simplify my code, thank you so much!