Queue with different samples_per_volume for each volume

[jmlipman]

Currently, I'm working with 2D slices of 3D images, and the number of slices per image can be very different, from 20 to 300. Therefore, setting Queue's parameter samples_per_volume seems tricky: e.g., 50 would probably be too much for certain images and too little for others. Is there any clever way to deal with this? Ideally, one would set sample_per_volume=number_of_2D_slices in combination to UniformSampler. To avoid this, I was tempted to use GridSampler, however, GridSampler does not seem to accept SubjectsDataset, which, in contrast to Subjects (that currently accepts), contains the data transformations that I want to apply.

Edit: Not sure if it makes sense with the current implementation but samples_per_volume could receive a list of ints s.t. its length = number of subjects within SubjectsDataset. If a single int is given, then extract the same samples per volume.

[fepegar]

Hi, @jmlipman. I think I understand the issue. I'm not sure how the grid sampler would help, though.

I think this is feasible. There are a couple of details that need to be taken into account carefully, such as making sure that the list of samples_per_volume is shuffled the same way as the subjects list. We would need to perform the shuffling manually, instead of letting the data loader do it internally.

Would you be willing to contribute for this?

[jmlipman]

I will write something here if I can come up with something nice and clean. I also had another small suggestion that those working with 2D slices would probably appreciate: squeezing the 3rd (dummy) dimension directly in Queue so that the data will come to the user with the right dimensions.

[jmlipman]

I think I managed. I had to add some object variables to keep track of the subjects, and I also randomized the subjects and the samples_per_volume list in the same way. I need to write some minimal-code tests to ensure the changes work well and when I have a bit of time I can send a pull request.

[Sheng-xc]

Hi @jmlipman, I came across your post about loading 2d slices of subjects with various samples_per_volume in a synchronized manner with GridSampler. Your approach sounds very elegant and aligns closely with what I am working on. If it's convenient for you, could you kindly share the details of your implementation?

[jmlipman]

Hi. Yes, I think it's this: https://github.com/jmlipman/SauronUNet/blob/main/torchio_lib/queue.py

[Sheng-xc]

Thank you for the help! It works very well.