Variable Metadata Extractor not working as expected #707
Description
What happened?
The ExtractVariableGroupAndLevel class allows the user to specify a shortcut (e.g. z) for including all pressure levels (z_50, z_100, z_500, ...). This shortcut can match the name of a surface variable (z). This results in inconsistent behaviour that will trigger an error in anemoi-training when pressure level scaling is used for that variable.
What are the steps to reproduce the bug?
from anemoi.training.utils.variables_metadata import ExtractVariableGroupAndLevel
from anemoi.datasets import open_dataset
ds = open_dataset("aifs-ea-an-oper-0001-mars-o96-1979-2023-6h-v8")
config = {
"default": "sfc",
"pl": {"param": ["q", "t", "u", "v", "w", "z"]},
"sfc": {"param": ["tp", "cp", "10u", "10v", "2d", "sp", "z"]}
}
extractor = ExtractVariableGroupAndLevel(config, ds.metadata()["variables_metadata"])
print(f"Variable: z, Group: {extractor.get_group('z')}")
print(f"Variable: t, Group: {extractor.get_group('t')}")
Version
main
Relevant log output
Variable: z, Group: pl
Variable: t, Group: plOrganisation
ECWMF