Loading GEFS forecasts 2017-2021 not working

[snath-xoc]

When trying to load GEFS forecasts (not reanalysis) from AWS for years before 2021 (i.e., 2017-2020 which should exist), Herbie cannot find the file. I find Herbie super useful so would be willing to put in some work to extend it to 2017, I just need some pointers on what would be the best place to start.

[williamhobbs]

What version of Herbie are you using? I think this was fixed/updated a few months ago in 2024.8.0. See #355 and #358.

[snath-xoc]

Strange

herbie.__version__

yields

'2024.8.0'

[snath-xoc]

I use the command

ds_20210101 = Herbie(
    "2017-01-01",
    model="gefs",
    fxx=36,
    product="atmos.25",
    member="mean",
).xarray("PRMSL").drop_attrs()

which returns

ValueError: product=atmos.25 not recognized. Must be one of dict_keys(['atmos.5', 'atmos.5b', 'atmos.25', 'wave', 'chem.5', 'chem.25'])

This does not happen otherwise with 2021-01-01 on

[williamhobbs]

It looks like the GEFS product you are looking for is available as far back as 2020-09-23, but not before that. Or, maybe it was labeled/stored a different way and the GEFS template in Herbie would need to be updated?

Herbie(
    "2020-09-23",
    model="gefs",
    fxx=36,
    product="atmos.25",
    member="mean",
)

returns

✅ Found ┊ model=gefs ┊ product=atmos.25 ┊ 2020-Sep-23 00:00 UTC F36 ┊ GRIB2 @ aws ┊ IDX @ aws
▌▌Herbie GEFS model atmos.25 product initialized 2020-Sep-23 00:00 UTC F36 ┊ source=aws

but

Herbie(
    "2020-09-22",
    model="gefs",
    fxx=36,
    product="atmos.25",
    member="mean",
)

returns

💔 Did not find ┊ model=gefs ┊ product=atmos.25 ┊ 2020-Sep-22 00:00 UTC F36
▌▌Herbie GEFS model atmos.25 product initialized 2020-Sep-22 00:00 UTC F36 ┊ source=None

[snath-xoc]

Hmmm yes you are right, the folder structure also changed and it seems like only the 1 degree products for pgrb2a and pgrb2b are available (even though this website indicates that the 0.25 should be available https://www.ncei.noaa.gov/products/weather-climate-models/global-ensemble-forecast#:~:text=The%20Global%20Ensemble%20Forecast%20System,observing%20systems%20biases%2C%20and%20the), perhaps better to use the reanalysis products then.

[snath-xoc]

Thank you for the help :)