Skip to content

catalogbuilder needs to be extended to handle static files for MDTF-diagnostics compatibility #118

@wrongkindofdoctor

Description

@wrongkindofdoctor

The catalogBuilder does not handle static files (e.g., ocean_monthly.static.nc) that contain cell area and other variables required by several PODs, including the MAR notebooks. intake-ESM handles multiple-varilbe files bit differently than single-variable files (https://intake-esm.readthedocs.io/en/stable/how-to/use-catalogs-with-assets-containing-multiple-variables.html), with the variable_id column populated with a list of file variables:

['SHF', 'REGION_MASK', 'ANGLE', 'DXU', 'KMT', 'TEMP', 'SiO3']

This paradigm does not allow metadata (e.g., standard_name) to be populated for each variable, so the following solution is suggested:

  1. if the file name contains the word 'static', use the 'long' method to open the file
  2. add a new catalog entry for each file variable with the metadata read from the file, and the frequency column populated with 'fx'. In this case, the file_path entry would be identical for each file variable. For example:
dev,,fre_om5,om5_b08,fx,ocean_monthly,,,,areacello,,5yr,gfdl.ncrc5-intel23-prod,<null>,,cell_area,/archive/John.Krasting/fre_om5/FMS2024.02_om5_20250206/om5_b08/gfdl.ncrc5-intel23-prod/pp/ocean_monthly/ocean_monthly.static.nc
dev,,fre_om5,om5_b08,fx,ocean_monthly,,,,deptho,,5yr,gfdl.ncrc5-intel23-prod,<null>,,sea_floor_depth_below_geoid,/archive/John.Krasting/fre_om5/FMS2024.02_om5_20250206/om5_b08/gfdl.ncrc5-intel23-prod/pp/ocean_monthly/ocean_monthly.static.nc

At this time, the workaround for MDTF functionality is to manually add entries for required variables to catalogs generated with the "slow" method.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions