-
Notifications
You must be signed in to change notification settings - Fork 6
Open
Labels
bugSomething isn't workingSomething isn't workingforcingIssue related to forcing (generation, loading, etc.)Issue related to forcing (generation, loading, etc.)
Description
When running the Makkink forcing generation for ERA5 for a range of a couple of years, it can use excessive memory causing the kernel to crash.
However, when defining ~/.esmvaltool/dask.yml as the following:
cluster:
type: distributed.LocalCluster
n_workers: 2
threads_per_worker: 1
memory_limit: 5 GiBIt runs fine with a memory use <2GB.
The issue probably occurs in the post-processing function where the potential evaporation is computed. However, we do run into conflicting advice:
- Dask recommends using the distributed scheduler.
- ESMValTool recommends the local sheduler as not all iris (and ESMF...) functionality works well with Dask Distributed.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingforcingIssue related to forcing (generation, loading, etc.)Issue related to forcing (generation, loading, etc.)