Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Analysis updates #11

Merged
merged 72 commits into from
Feb 3, 2025
Merged

Analysis updates #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
344183b
Remove old and unused bin/ scripts
Jul 12, 2024
53a76b6
cylc gui can be run anywhere now
Jul 12, 2024
7bdbeca
Removed pp.yaml updates that are now in fre-cli
Jul 12, 2024
33f81f7
Remove unused and broken mdtf task. mdtf can be run as an analysis sc…
Jul 12, 2024
38d31fc
#147 Insist the data-catalog task succeed
Jul 12, 2024
a8f0904
#147 build the catalog after saving chunk-b timeseries
Jul 16, 2024
693ad9b
#147 Generate data catalog before running analysis scripts
Jul 16, 2024
a7b5d98
#147 Build analysis script environments, cylc graph and task definit…
Jul 16, 2024
6c0fc4a
#147 fix; add build-analysis- tasks for the every-interval and defin…
Jul 16, 2024
b83d42e
#147 fix; add the data-catalog graph for the cumulative and every-in…
Jul 16, 2024
5eaba04
#147 .csh for csh, .fish for fish, but nothing for bourne
Jul 17, 2024
ed27ad9
#147 add analysis publish tasks for cumulative analysis
Jul 17, 2024
0e524e3
Merge remote-tracking branch 'gh/main' into 147.catalogs-before-analysis
Aug 19, 2024
67c2562
#147 Added initial cycle point dependence with ^ syntax
Aug 22, 2024
06b8dcc
#147 dummy publish-analysis environment
Aug 22, 2024
c015dd6
#147 Put the build tasks in R1 and put the build dependence in the a…
Aug 30, 2024
8fe660c
Turn off epmt collection while epmt service is down
Aug 30, 2024
b797dc1
#147 Plausible updates to graph and define build and publish tasks
Sep 4, 2024
f01332f
#147 Remove the draft analysis-install script from flow.cylc
Sep 4, 2024
8b4f12d
#147 Add install ray script
Sep 4, 2024
6109f75
Merge branch 'main' into 147.catalogs-before-analysis
ceblanton Sep 23, 2024
a3cbe77
#147 For clarity print to stderr whether the build and publish steps…
Sep 23, 2024
fb5881a
Update the soon-to-be-retired app/analysis/rose-app.conf analysis col…
Sep 23, 2024
5ac4301
#147 Use the correct Cylc job env variable
Sep 23, 2024
6f05cb0
#147 add dummy publish script
Sep 23, 2024
5e85296
Rename ray-analysis to cloud
Sep 23, 2024
20fd037
#137 analysis-clouds to use the cylc-installed python environment
Sep 23, 2024
6a3063e
#147 add plausible new land analysis script from Anthony
Sep 23, 2024
d61275e
Set the stall timeout to 1 week; the default of one hour is way too s…
Sep 30, 2024
8da5ddf
#147 include the full path to the publish script
Sep 30, 2024
6b9ddfe
#147 Added rewritten analysis jinja methods
Oct 23, 2024
9819b9d
#147 Update flow.cylc to use new get_analysis_info interface
Oct 29, 2024
cf437f3
#147 updated interface to get_analysis_info
Oct 30, 2024
295a117
#147 various fixes for rewritten get_analysis_info.py; passes cylc v…
Oct 30, 2024
26b2edf
Merge remote-tracking branch 'origin/main' into 147.catalogs-before-a…
Dec 17, 2024
1964606
Remove analysis scripts from repository
Dec 17, 2024
f199d90
Retrieve the combined yaml as <experiment>.yaml
Dec 17, 2024
30c2bbe
Remove analysis script configuartions from fre-workflows
Dec 21, 2024
51c944c
keep $FRE_ANALYSIS_HOME available to analysis scripts in a hard-coded…
Dec 21, 2024
6e27d73
Parse time consistently, and some wip changes
Dec 21, 2024
2447648
added install graph and definitions for one case
Dec 21, 2024
9857d21
Task graph for new-analysis cases every-chunk every-chunk-cumulative
Dec 21, 2024
7e20986
New analysis graphs should be working now
Dec 23, 2024
0dfd355
Graph updates for the 3 types of new-analysis scripts,
Jan 6, 2025
225dfcb
Various fixes for legacy analysis scripts mostly
Jan 7, 2025
1ca84c5
Thanks to Ray's get_analysis_info updates, we can reduce these
Jan 7, 2025
dfcc56a
Set inherited scripting for analysis-build tasks, and hard-code FRE-V…
Jan 8, 2025
ad2b0cc
Fix typo on line continuation in system call
Jan 8, 2025
241b4a1
Replace "canopy" with {{ FRE_VERSION }}
Jan 8, 2025
64df473
Retry 'fre analysis build' tasks twice as it fails sometimes for unkn…
Jan 8, 2025
00db166
plausible task definitions for new and legacy analysis scripts
Jan 8, 2025
b0bf95e
identation fixes
Jan 8, 2025
94af7b5
ready for runtime testing
Jan 8, 2025
6c9f482
Remove default rose scripting for tasks that do not set script or env…
Jan 13, 2025
2b5ccd2
analysis tasks should load fre
Jan 15, 2025
a4fe833
Updates for the defined-interval scripting
Jan 15, 2025
ad85a03
Various fixes
Jan 21, 2025
23527ea
Various fixes for analysis scripts
Jan 21, 2025
13bfe1d
More legacy analysis adjustments
Jan 24, 2025
b42691b
a few more legacy analysis changes
Jan 24, 2025
6d10f8d
Add check for invalid analysis timeaverage requests.
Jan 24, 2025
7b60839
Hard fail if analysis script requests non-available component, and ad…
Jan 25, 2025
a30c29f
Some analysis scripts expect a trailing slash for $in_data_dir
Jan 27, 2025
b64850b
Add databegyr/dataendyr legacy analysis template vars.
Jan 27, 2025
28ca4bd
The analysis script jinja call-out must have access to both pp chunks,
Jan 29, 2025
f4fa958
Update get-analysis-info callout to choose between the available pp c…
Jan 29, 2025
01ed4be
Workaround for data catalog race condition
Jan 29, 2025
3343cd1
Retry new analysis scripts 10 times until we solve the intake-esm pro…
Jan 31, 2025
fc835fc
Merge remote-tracking branch 'origin/main' into 147.catalogs-before-a…
Jan 31, 2025
5681587
Remove unintended changes to regrid-xy and remap-pp-component rose-ap…
Jan 31, 2025
6796c04
Undo unintended change to rose-suite.conf template
Jan 31, 2025
30f8b8c
Merge remote-tracking branch 'me/147.catalogs-before-analysis' into 1…
Jan 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,055 changes: 590 additions & 465 deletions Jinja2Filters/get_analysis_info.py
View file
Open in desktop

Large diffs are not rendered by default.

160 changes: 0 additions & 160 deletions app/analysis/file/atw_atmos_av_annual_column.csh
View file
Open in desktop

This file was deleted.

161 changes: 0 additions & 161 deletions app/analysis/file/atw_atmos_ts_monthly_index.csh
View file
Open in desktop

This file was deleted.

Loading