Skip to content

Validation fails silently in Anemoi training when time-range does not span the data #511

@manvendra9099

Description

@manvendra9099

Describe the bug
We found that validation doesn't run at all (without any warning/error) when dataloader.validation.start/end dates don't overlap with the actual data in the dataset

Example:

dataloader:
  validation:
    start: 2020-07-01 00:00:00   # <-- worked
    # start: 2021-01-01 00:00:00 # <-- silently skips validation
    end: 2020-07-31 23:59:00 
    # end: 2021-12-31 23:59:00

Run training with this config. The validator does not run and logs no validation metrics or warnings.

Actual behaviour:
Training runs normally
No validation occurs
No warning or error is logged

Expected behavior:
When the validation range yields zero samples: Throw an error or at least a warning early in the dataloader setup
Include in the message:

  • requested validation.start/end
  • dataset actual coverage (tmin/tmax)

Recommended fix
Raise a ValueError in _dates_to_indices if the requested date range yields zero samples, with a message showing the requested and dataset date ranges.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    To be triaged

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions