Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

let fre cmor run process tripolar ocean data #303

Merged
merged 50 commits into from
Jan 16, 2025

Conversation

ilaflott
Copy link
Member

@ilaflott ilaflott commented Dec 24, 2024

this PR will ideally enable fre cmor run to process ocean data on a native tripolar grid. also will do some general clean up here and there. relevant background: #268 PCMDI/cmor#766 and PCMDI/cmor#767

Describe your changes

  • remove GitPython pip-only dependency that can only be fulfilled by terrible habits
  • add PCMDI/cmip6-cmor-tables as submodule dependency
  • set minimum python version to 3.9 minimum
  • remove redundant click context forwarding in fre cmor
  • remove redundant click context forwarding in most places, exception: fre catalog
  • re-set os.environ['HOME'] back to inital value in test_fre_make_cli.py
  • puts meta.yaml source: url: <this repo's URL> back to source: path: . in light of pipeline-error-induced insanity.
  • pylint edits to address scary-ish no-value-for-parameter errors revealed by adjusting click scaffolding
  • NotImplemented errors for fre functions that do not yet have any real guts to them in fre-cli
  • get rid of sys.exit where possible
  • removes "scratchwork" style fre cmor scripts used for by-hand local testing
  • add's gfdl/ppan-only tests in fre/cmor/tests/test_cmor_run_subtool_ppan_only.py that gracefully fail when not at pp/an with an appropriate warning
  • tweak pipeline comments
  • make pylint kinder to long lines (--max-line-length 100 --> 120)
  • remove unneeded __init__.py files in testing directories ... added back in, affected coverage stats
  • last fre cmor run case of original 9 (Omon / sos / gn) case succeeds now

Issue ticket number and link (if applicable)

will close #298

Checklist before requesting a review

  • I ran my code
  • I tried to make my code readable
  • I tried to comment my code
  • I wrote a new test, if applicable
  • I wrote new instructions/documentation, if applicable
  • I ran pytest and inspected it's output
  • I ran pylint and attempted to implement some of it's feedback

…on pip-only dependency... nasty tech debt i should get rid of before its too late!
@ilaflott ilaflott added bug Something isn't working new functioning New feature or request tricky Likely to encounter significant friction to solution priority: HIGH labels Dec 24, 2024
@ilaflott ilaflott self-assigned this Dec 24, 2024
@ilaflott ilaflott linked an issue Dec 24, 2024 that may be closed by this pull request
9 tasks
@ilaflott
Copy link
Member Author

so asking conda/mamba to use python3.10 or greater is surprisingly an issue. Dinking around with this locally, the first thing it complains about is cylc-uiserver. Given that this has yet-to-be-utilized in the actual repository, I comment it out.

The next issue is catalogbuilder... actually, really, a requirement of catalogbuilder's. intake-esm==2023.7.7, which seems to desire a python of 3.9 or less, but 3.9 is nearing EOL.

Given that this quick python version bump was supposed to be a quick-sidequest, i'll be restoring the dependencies and barking up that tree some other time.

…aving cmor>=3.9.0 because i know we should not bother with anything less than 3.9.0 for cmor
@ilaflott
Copy link
Member Author

ilaflott commented Dec 24, 2024

every now and then, a random conda build fails... here, it claims import git is causing the failure. I did remove that module, but i also removed that line

edit: same failure, same job. sanity demanding i ignore the conda_build workflow for a minute apparently

edit2: same failure, different job. xmas nightmare! moving on...

ilaflott and others added 5 commits December 24, 2024 13:38
cmor_mixer now creates xh and yh bnds. this was not supposed to take so long... why? see note-to-self...

...note to self, if you need to write data to a netcdf file... don't use [x][y][z] style indexing. you HAVE to use [x,y,z] for the assignment or the assignment fails.
@ilaflott ilaflott marked this pull request as ready for review January 9, 2025 23:27
Copy link
Collaborator

@ceblanton ceblanton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good but admittedly I did not try it and review hard. In the future we will do more rigorous reviewing, promise!

@ceblanton ceblanton merged commit 3453c6b into main Jan 16, 2025
2 checks passed
@ceblanton ceblanton deleted the 298-cmor-process-ocean-tripolar-grid branch January 16, 2025 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working new functioning New feature or request priority: HIGH tricky Likely to encounter significant friction to solution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CMORmeal: successfully process ocean variables on tripolar grid with cmor
2 participants