Add multiple images to channel

[vijaypshah]

Hi,

I have images stored in multiple files. I would like to load them but append each image to color channel.
I can make different type in Subject and load them. However, not sure how I can append them to color channel.

e.g. Subject_a = tio.Subject( A=tio.ScalerImage("A.jpg"),B=tio.ScalerIamge("B.img"))
But now I would like A and B images to be color channel. Assuming that A and B images are of same size.

Best Regards,
Vijay

[fepegar]

The docs for Image say:

If a sequence of paths is given, data will be concatenated on the channel dimension so spatial dimensions must match.

Therefore, it should work if you do something like:

subject_a = tio.Subject(image=tio.ScalarImage(["A.jpg", "B.img"]))

[vijaypshah]

Great Thanks!

If my images are not in same geomerty, andwould like to first apply some transformation before concate them to channel.
Would it be something that need to be written as Lambda transformation function ?

Best Regards,
Vijay

[fepegar]

Can you tell me more about your use case? Why do you want to concatenate your images along the channel dimension? What kind of images are you using and what's the task?

[vijaypshah]

In multi- imaging sometimes the affine matrix may not be the same. Some of the example you have all images are preprocess, so I can see one could pre-process all images and then load like you indicated to concate in the channel.

I am just experimenting as of now loading the images without having to do pre-processing in separate steps. Your package seems ideal to accomplish those tasks.

Task:
Load image A, B, C [different dimension and affine matrix]
Make all images of same spacing and affine matrix
concate them in channel if required.

[fepegar]

Suppose you have two images from different modalities:

T1 MRI:

T2 MRI:

These images clearly have different spacings and sizes but are registered.

import torchio as tio
subject = tio.Subject(
    t1=tio.ScalarImage('t1.nii.gz'),
    t2=tio.ScalarImage('t2.nii.gz'),
)
subject.plot()

You can resample the second one in the space of the first:

resample = tio.Resample('t1')
resampled = resample(subject)
resampled.plot()

In [14]: subject.t1
Out[14]: ScalarImage(shape: (1, 193, 229, 193); spacing: (1.00, 1.00, 1.00); orientation: RAS+; memory: 32.5 MiB; dtype: torch.FloatTensor)

In [15]: subject.t2
Out[15]: ScalarImage(shape: (1, 193, 37, 193); spacing: (1.00, 3.00, 1.00); orientation: RAS+; memory: 5.3 MiB; dtype: torch.FloatTensor)

In [16]: resampled.t1
Out[16]: ScalarImage(shape: (1, 193, 229, 193); spacing: (1.00, 1.00, 1.00); orientation: RAS+; memory: 32.5 MiB; dtype: torch.FloatTensor)

In [17]: resampled.t2
Out[17]: ScalarImage(shape: (1, 193, 229, 193); spacing: (1.00, 1.00, 1.00); orientation: RAS+; memory: 32.5 MiB; dtype: torch.FloatTensor)

You can always create your own transform to concatenate them along the channel dimension, but I would just do that before feeding them to the network:

In [18]: dataset = tio.SubjectsDataset([subject], transform=resample)

In [19]: import torch

In [20]: loader = torch.utils.data.DataLoader(dataset)

In [21]: batch = iter(loader).next()

In [23]: inputs = torch.cat((batch['t1'][tio.DATA], batch['t2'][tio.DATA]), dim=1)

In [24]: inputs.shape
Out[24]: torch.Size([1, 2, 193, 229, 193])

[vijaypshah]

Great thank you @fepegar for detailed explaination!
Very helpful!

Best Regards,
Vijay