Update CRU CMORizer

[lukruh]

Description

This is a new approach to update the CRU CMORizer. It includes fixes for the time coordinate, adds the number of stations per cell and provides backwards compatible support for multiple versions of the dataset (TS4.02, TS4.06, TS4.07).

• fix time coord (from Cmorize cru v4.04 & stn contraint #1981)
• add evspsblpot (not in recipe_check_obs.yml yet)
• update config and formatter to match WOA multi version style
• add versions 4.07 and 4.06
• save number of stations via AuxCoord (not compatible to all preproc functions)
• update references in datasets.yml (set to newest only)
• add more variables (evspsblpot, tasmin, tasmax, clt)
• test formatter and recipes for 4.02
• test affected recipes
• • recipe_anav13jclim.yml
• • recipe_martin18grl.yml
• • schlund20jgr/recipe_schlund20jgr_gpp_change_rcp85.yml
• • schlund20jgr/recipe_schlund20jgr_gpp_abs_rcp85.yml

Closes #3380

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🛠 This pull request has a descriptive title
• 🛠 Code is written according to the code quality guidelines
• 🛠 Documentation is available
• 🛠 Tests run successfully
• 🛠 The list of authors is up to date
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 All checks below this pull request were successful

New or updated data reformatting script

• 🛠 Documentation is available
• 🛠 The dataset has been added to the CMOR check recipe
• 🛠 The dataset has been added to the shared data pools of DKRZ and Jasmin by the @ESMValGroup/OBS-maintainers team
• 🧪 Numbers and units of the data look physically meaningful

---

To help with the number of pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[lukruh]

Regarding support of multiple version support: The cmorizer is still able to reproduce the same cmorized data for v TS4.02 as before. However, changes to (downloader, ) datasets.yml and cmor_config/CRU.yml are required to do so. I'm not completly happy with it, but I hope its fine for the scope of this PR. (opened #3430)

Regarding codacy errors: Line lengths errors are caused by long urls, which I prefer not to break, but I can if required. The unused arguments seem to be required by the cmorizer class that imports the function. There are several formatters ignoring them, should we consider to make them optional?

[lukruh]

I'm thinking about throwing the number of stations coordinate out again. I don't even know if anyone would use it and it might clash with some simple simple iris operations like merging or substracting different parts of a cube.

[rbeucher]

@flicj191 when you have some time, can you look at this please?

[flicj191]

thanks @lukruh, from my experience you don't have to worry about the unused arguments codacy errors, that's been like that for a while. I have used this and downloaded and cmorized cru data on Gadi (Australia). It was picked up fine and comparable in an edited version of recipe_anav13jclim.yml
Data is on Gadi though I can't do that on DKRZ

[mo-gill]

Hi, we are currently working on the ESMValTool release for v2.11.0. We're wondering if you'd be able to complete this PR by the end of next week (Friday 10th May).

Otherwise, please let us know, and we'll move it into the next milestone for you 🙂

[rbeucher]

Looks good on my side.

[rbeucher]

Circle CI returns some formatting issues:

./esmvaltool/cmorizers/data/formatters/datasets/cru.py:7:80: E501 line too long (86 > 79 characters)
./esmvaltool/cmorizers/data/formatters/datasets/cru.py:8:80: E501 line too long (86 > 79 characters)
./esmvaltool/cmorizers/data/formatters/datasets/cru.py:9:80: E501 line too long (86 > 79 characters)
./build/lib/esmvaltool/cmorizers/data/formatters/datasets/cru.py:7:80: E501 line too long (86 > 79 characters)
./build/lib/esmvaltool/cmorizers/data/formatters/datasets/cru.py:8:80: E501 line too long (86 > 79 characters)
./build/lib/esmvaltool/cmorizers/data/formatters/datasets/cru.py:9:80: E501 line too long (86 > 79 characters)

Exited with code exit status 1

[lukruh]

Circle CI returns some formatting issues:

./esmvaltool/cmorizers/data/formatters/datasets/cru.py:7:80: E501 line too long (86 > 79 characters)
./esmvaltool/cmorizers/data/formatters/datasets/cru.py:8:80: E501 line too long (86 > 79 characters)
./esmvaltool/cmorizers/data/formatters/datasets/cru.py:9:80: E501 line too long (86 > 79 characters)
./build/lib/esmvaltool/cmorizers/data/formatters/datasets/cru.py:7:80: E501 line too long (86 > 79 characters)
./build/lib/esmvaltool/cmorizers/data/formatters/datasets/cru.py:8:80: E501 line too long (86 > 79 characters)
./build/lib/esmvaltool/cmorizers/data/formatters/datasets/cru.py:9:80: E501 line too long (86 > 79 characters)

Exited with code exit status 1

@rbeucher Thanks for the review!
These are URLs to the source file. I personally prefer not to break them. With # noqa: E501 I can ignore the error locally but not for codacy. If you think I should break them I will do it. Can I consider your review as a technical or a scientific one?

@mo-gill From my side it could be merged, after a second review. If it's already to late feel free to move it to the next mile stone.

[rbeucher]

Yes I would avoid breaking them. Technical review.

[bettina-gier]

Just change the version in the cmorize_obs recipe and all is good. Data in new version looks good, everything runs fine, small changes in recipe outputs, but that's what new versions come with - all scientifically plausible.
@ESMValGroup/obs-maintainers if Levante people want to just move the data from my folder to the public one:
raw: /work/bd0854/b309137/data_ext/RAW/Tier2/CRU
cmorized: /work/bd0854/b309137/data_ext/Tier2/CRU

[lukruh]

Thanks for the review @bettina-gier. Good spot of v4.06 in the test recipe. I downloaded and tested both versions, the most recent one should stay in the test recipe of cause.

@rbeucher since you are fine with not breaking the urls, could you withdraw the change request and approve it again? I think with open change request it can not be merged. Thanks a lot.

[hb326]

@lukruh and @bettina-gier: we have copied the data to the respective folders on Levante. All sorted.
@bettina-gier: you can delete them now from your work folder.