Create module to tar post-processed PS-OCT data

[kabilar]

Based on today's discussion, we will package the post-processed files (listed below) for all tiles in a given slice into a tar file. This post-processing is performed on the acquisition system and these files are generally ~0.5 MB in size.

1. mosaic_005_image_001_processed_aip.nii
2. mosaic_005_image_001_processed_mip.nii
3. mosaic_005_image_001_processed_orientation.nii
4. mosaic_005_image_001_processed_retardance.nii
5. mosaic_005_image_001_processed_surface_finding.nii

In the future we will convert these files to Zarr, and delete these tar files.

cc @calvinchai @balbasty @ayendiki @satra

[satra]

i thought processing takes 4 hours. if so i would tar and send the unprocessed files.

[kabilar]

Hi @satra, there are a few pieces that we discussed during the computational meeting that are not captured here. Pre-processing happens on the fly on the acquisition system, which generates a ~1 GB nifti for each tile. Approximately 1,700 files are created for each slice by the pre-processing pipeline. Since each pre-processed nifti file is ~1 GB we will not be tar-ing them since these are not small files and there is not enough space on the acquisition system to create a tar for each slice. We will upload these pre-processed tiles as each slice is acquired.

Post-processing is what takes longer (~4 hours). Eventually we can run post-processing elsewhere but for the short-term we will continue to run it on the acquisition system.