Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow noncontinous time ranges in datasets for training and validation #128

Open
HCookie opened this issue Nov 8, 2024 · 0 comments · May be fixed by #129
Open

Allow noncontinous time ranges in datasets for training and validation #128

HCookie opened this issue Nov 8, 2024 · 0 comments · May be fixed by #129
Labels
enhancement New feature or request

Comments

@HCookie
Copy link
Member

HCookie commented Nov 8, 2024

Is your feature request related to a problem? Please describe.

Currently training, validation and test datasets are continuous time ranges. While this makes sense, it may be interesting to explore other ways of organising the time slices.

Describe the solution you'd like

Add a way to provide ranges of time to the dataloader, allowing the following config.

training:
  dataset: ${dataloader.dataset}
  ranges: 
    - [1970, 1980]
    - [1990, 2020]
  frequency: ${data.frequency}
  drop:  []


validation:
  dataset: ${dataloader.dataset}
  ranges: 
    - [1981, 1989]
    - [2021, 2021]
  frequency: ${data.frequency}
  drop:  []

Describe alternatives you've considered

No response

Additional context

No response

Organisation

ECMWF

@HCookie HCookie added the enhancement New feature or request label Nov 8, 2024
@HCookie HCookie linked a pull request Nov 8, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant