Skip to content

Add support for compressing mask npz files #200

@aaron-hopkinson

Description

@aaron-hopkinson

Feature request from @elkir

Originally posted by @elkir in #108 (comment)

Any chance to add support for compressing the npz files?

   def make_global_on_lam_mask(lam_lat, lam_lon, global_lat, global_lon, output, **kwargs):
   
      # Extract compression and dtype options
      compress = kwargs.pop('compress', False)
      
      ...
      
      # Save the mask with or without compression
      if compress:
          np.savez_compressed(output, mask=mask)
      else:
          np.savez(output, mask=mask)
          
          
   ...
   
   
      class MakeGlobalOnLamMask:
           ...
           command_parser.add_argument(
               "-z", "--compress",
               action="store_true",
               help="Enable compression for the output file.",
           )
           ...
        def run( ...
            ...
            make_global_on_lam_mask(...
                     compress=args.compress,
            )

They are quite big for int64, and yet very compressible.

The regrid filter already works with compressed npz file as is, I've tested it for manually compressed ones.

This will make it easier to store right next to recipes in git repos.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    To be triaged

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions