Skip to content

Forecast output needs to be written incrementally #1596

@clessig

Description

@clessig

What happened?

Currently, forecast output is written after all forecast steps have been completed. This limits the number of steps we can consider, especially at very high resolutions. Hence, we should have an inference mode where the output is written after each step, or a small number of steps. (I cannot see a reason to have this in training, but it might exist). The current model.forward() function can be used--and should, to make sure we run training and inference identically-- but some minor modifications are needed. This will likely also affect the validation_io code, since we need to make sure the data is appended correctly.

@ankitpatnala @SavvasMel @Jubeku @grassesi

What are the steps to reproduce the bug?

No response

Hedgedoc link to logs and more information. This ticket is public, do not attach files directly.

No response

Metadata

Metadata

Assignees

Labels

data:ioIssues with the zarr output produced during inference/validationinitiativeLarge piece of work covering multiple sprintmodelRelated to model training or definition (not generic infra)model:inferenceanything related to the inference step (not plotting or score computation).

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions