Conversation
- Simplified the extraction of variable mappings from the CMIP7 Data Request - Removed out_name usages
Codecov Report❌ Patch coverage is
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
@bouweandela this will cause duplicate test data now following all of the DRS |
packages/climate-ref-core/src/climate_ref_core/cmip6_to_cmip7.py
Outdated
Show resolved
Hide resolved
| attrs["realization_index"] = convert_variant_index(attrs.get("realization_index", 1), "r") | ||
| attrs["initialization_index"] = convert_variant_index(attrs.get("initialization_index", 1), "i") | ||
| attrs["physics_index"] = convert_variant_index(attrs.get("physics_index", 1), "p") | ||
| attrs["forcing_index"] = convert_variant_index(attrs.get("forcing_index", 1), "f") |
There was a problem hiding this comment.
Somehow my data always seems to end up with f1, even when the input data has f2
There was a problem hiding this comment.
Fall back to using value parsed from variant label, but it could also because convert_variant_index explicitly checks for int so numpy ints fail. Either way its more defensive now
bouweandela
left a comment
There was a problem hiding this comment.
I compared the results with what CMOR generates. It is fairly similar, except for the issue with the name of the primary variable.
I also noticed that the output from the REF contains a scalar height coordinate on the bounds variables, which does not look right. For example:
netcdf tas_tmaxavg-h2m-hxy-u_mon_glb_gn_UKESM1-0-LL_historical_r1i1p1f2_185001-194912 {
dimensions:
time = UNLIMITED ; // (1200 currently)
bnds = 2 ;
lat = 144 ;
lon = 192 ;
variables:
double time_bnds(time, bnds) ;
time_bnds:_FillValue = NaN ;
time_bnds:coordinates = "height" ;
...
double height ;
height:_FillValue = NaN ;
height:units = "m" ;
height:axis = "Z" ;
height:positive = "up" ;
height:long_name = "height" ;
height:standard_name = "height" ;
note the line time_bnds:coordinates = "height" ;
Two bugs caused non-r1i1p1f1 datasets to always produce r1i1p1f1:
1. convert_variant_index() used isinstance(value, int) which missed
numpy integer types (np.int32, np.int64) from netCDF attributes,
falling through to the default return of f"{prefix}1"
2. convert_cmip6_to_cmip7_attrs() defaulted all missing individual
indexes to 1 and rebuilt variant_label from those defaults,
overwriting the correct variant_label even when it was present
Fix: parse variant_label as fallback when individual indexes are
missing, and handle numpy integer types in convert_variant_index().
|
I learnt something about xarray today with that spurious bound! |
* origin/main: (86 commits) Bump version: 0.11.0 → 0.11.1 docs: add changelog for #567 fix: reduce memory during ingestion and add solve logging fix: prevent DRS re-ingestion from regressing finalised datasets Bump version: 0.10.0 → 0.11.0 chore: Update comment chore: upgrade pins for ilamb fix: revert compat=override on open_mfdataset docs: add changelog for #565 chore: Upgrade lockfile and fix some errors chore: add coverage chore: add default separator in alembic fix: time_coder warning chore: Pin to use tas fix(solver): preserve DataCatalog wrapper in apply_dataset_filters fix(tests): use to_frame() when accessing DataCatalog in solver tests docs: Changelog chore: run the finalise in threads chore: clean up chore: add fix changelog entry for PR #561 ...
Description
Checklist
Please confirm that this pull request has done the following:
changelog/