Evaluation of model on up-sampled and down-sampled images

[plbenveniste]

I am working on up-sampling and down-sampling the test images on which I will evaluate the performance.
To up-sample and down-sample the images, I am using the following for instance:

sct_resample -i image.nii.gz -o output.nii.gz -f 1.1

First question:

By doing this, my initial image goes from a weight of 17.5 MB to 105.2 MB. Should I perform an additional operation to reduce its weight? Something like sct_image ... -type float64?

Second question:

I am also wondering what is the best strategy for evaluating the prediction on this up-sampled image?
I am considering 2 options:

• Option 1: upsample the ground truth as well using the following: sct_resample -i label.nii.gz -o output_label.nii.gz -f 1.1 -x nn
• Option 2: I downsample the prediction to the original resolution: sct_register_multimodal -i pred.nii.gz -d initial_image.nii.gz -o reg_label.nii.gz -identity 1 -x nn

I would be interesting in having your opinions on this.
Thank you for your help

Tagging @joshuacwnewton @jcohenadad @valosekj

[jcohenadad]

Something like sct_image ... -type float64

float64 will likely not reduce the weight-- i suspect the original is float32, so you could try that

[jcohenadad]

Option 2: I downsample the prediction to the original resolution

I'd rather do that, to compare with the same amount of voxels in the target (reference) object. As you know, the more voxels in the object, the higher the Dice (for the same overlap)

[plbenveniste]

Thanks for your feedback! 😃

[joshuacwnewton]

I am working on up-sampling and down-sampling the test images on which I will evaluate the performance.

Could you point to a sample image just for reference? I'm curious about the dimensions/contents/etc. before and after the resampling.

Should I perform an additional operation to reduce its weight?

I recall when trying to compress images in the past that nibabel doesn't apply maximum compression efficiency by default, so you can always un-gzip and then gzip again to save a little more space (at the cost of taking a little longer to load an image, but this is OK for segmentation -- this is only relevant for scripts like moco that do a lot of file opening/closing).

            subprocess.run(["gunzip", target_path], check=True)
            uncompressed_name = target_path[:-3]  # Remove .gz
            subprocess.run(["gzip", "-9", uncompressed_name], check=True)

Not sure how much of a difference this will make for images that are hard to compress (e.g. ones without many 0 voxels).

[plbenveniste]

Could you point to a sample image just for reference? I'm curious about the dimensions/contents/etc. before and after the resampling.

I have uploaded an image on duke here ~/duke/temp/plben/ms-lesion-agnostic/temp_resampling. It contains the following:

• the image: image.nii.gz: it weights 17.2 MB
• the image saved with float32: image_float32.nii.gz: I got it using sct_image -i image.nii.gz -o image_float32.nii.gz -type float32: it weights 22.6 MB
• the resampled image: output_resampled_factor_1p1.nii.gz: I got it using sct_resample -i image.nii.gz -o output_resampled_factor_1p1.nii.gz -f 1.1: it weights 104.6 MB
• the resampled image saved with float32: output_resampled_float32.nii.gz: I got it with sct_image -i output_resampled_factor_1p1.nii.gz -o output_resampled_float32.nii.gz -type float32: it weights 49.5 MB

I recall when trying to compress images in the past that nibabel doesn't apply maximum compression efficiency by default, so you can always un-gzip and then gzip again to save a little more space (at the cost of taking a little longer to load an image, but this is OK for segmentation -- this is only relevant for scripts like moco that do a lot of file opening/closing).

        subprocess.run(["gunzip", target_path], check=True)
        uncompressed_name = target_path[:-3]  # Remove .gz
        subprocess.run(["gzip", "-9", uncompressed_name], check=True)

Regarding this, I did the following on both float32 files:

cp image_float32.nii.gz image_float32_gzip.nii.gz
gzip -d image_float32_gzip.nii.gz
gzip -9 image_float32_gzip.nii

The results are the following:

• image_float32_gzip.nii.gz now weights 20.6 MB
• output_resampled_float32_gzip.nii.gz: now weights 49.6 MB

The improvement is pretty limited! Is this something we want to look deeper into for SCT?

Here is the summary of the weights:

drwxr-xr-x 2 p119007@GRAMES.POLYMTL.CA domain users@GRAMES.POLYMTL.CA      4096 jui 23 03:52 ./
drwxr-xr-x 2 p119007@GRAMES.POLYMTL.CA domain users@GRAMES.POLYMTL.CA      4096 jui 23 03:28 ../
-rw-r--r-- 1 p119007@GRAMES.POLYMTL.CA domain users@GRAMES.POLYMTL.CA  20613046 jui 23 03:50 image_float32_gzip.nii.gz
-rw-r--r-- 1 p119007@GRAMES.POLYMTL.CA domain users@GRAMES.POLYMTL.CA  22638476 jui 23 03:48 image_float32.nii.gz
-rw-r--r-- 1 p119007@GRAMES.POLYMTL.CA domain users@GRAMES.POLYMTL.CA  17217368 avr 15 06:55 image.nii.gz
-rw-r--r-- 1 p119007@GRAMES.POLYMTL.CA domain users@GRAMES.POLYMTL.CA 104616892 jui 23 03:47 output_resampled_factor_1p1.nii.gz
-rw-r--r-- 1 p119007@GRAMES.POLYMTL.CA domain users@GRAMES.POLYMTL.CA  49645602 jui 23 03:51 output_resampled_float32_gzip.nii.gz
-rw-r--r-- 1 p119007@GRAMES.POLYMTL.CA domain users@GRAMES.POLYMTL.CA  49535383 jui 23 03:48 output_resampled_float32.nii.gz

[joshuacwnewton]

Somehow the resampled output is larger after re-gzipping... darn. 😓

I'm happy that converting to float32 helped quite a bit though!