Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sub-hourly support to grib_to_netcdf #219

Open
wants to merge 10 commits into
base: develop
Choose a base branch
from

Conversation

joobog
Copy link
Collaborator

@joobog joobog commented Jun 17, 2024

No description provided.

@shahramn shahramn self-assigned this Jun 17, 2024
@shahramn shahramn added the approved-for-ci Approved to run CI on ECMWF machines label Jun 17, 2024
@iainrussell
Copy link
Member

iainrussell commented Jun 17, 2024

Hi @joobog,
Could you also test this with this GRIB file please?
https://github.com/ecmwf/cfgrib/blob/master/tests/sample-data/step_60m.grib
It's interesting because the steps are 0m, 60m, 120m, etc, which ecCodes decodes as step=0m,1h,2h - i.e. the units change between fields.

@shahramn
Copy link
Collaborator

Thanks @iainrussell
With that step_60m.grib file, the conversion actually fails!

> grib_to_netcdf -ox.nc step_60m.grib
grib_to_netcdf: Version 2.36.0
grib_to_netcdf: Processing input file 'step_60m.grib'.
grib_to_netcdf: Found 73 GRIB fields in 1 file.
grib_to_netcdf: Ignoring key(s): method, type, stream, refdate, hdate
grib_to_netcdf: Creating netCDF file 'x.nc'
grib_to_netcdf: NetCDF library version: 4.9.2 of May  1 2023 09:07:13 $
grib_to_netcdf: Creating large (64 bit) file format.
ECCODES ERROR   :  Cannot handle fields for different stepUnits.

ECCODES ERROR   :  Please split input data into different files. Exiting!

@joobog
Copy link
Collaborator Author

joobog commented Jun 18, 2024

Hmm, step_60m.grib does indeed show an interesting case. The units in sub-hourly are determined dynamically. This means that to get grib_to_netcdf right, we have to scan the entire file and select the smallest unit (which is a costly operation), or we can specify the time always in seconds, which is the smallest unit supported by grib_to_netcdf.

@iainrussell
Copy link
Member

Or perhaps use the keys that return the step in the units it was originally encoded in? I guess it would generally be the case that for a given GRIB file all the messages would be encoded with the same step units. The problem only arises because we try to convert the units (unless the step is zero).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved-for-ci Approved to run CI on ECMWF machines
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants