RFC: organize model outputs in subdirs during model runtime #7959
Description
a popular tool used to organize outputs is built into cime; it is short-term archive (or st_archive for short). To explain what that tools does, consider a case $CASE:
$CASE> tree
.
├── build
│ ...
├── case_scripts
│ ├──
│ ├── case.st_archive
│ ├── CaseStatus
│ ├── case.submit
│ ...
├── run
│ ...
│ $CASE.$CMP.HISTORY_STREAM.DATE.nc
│ $CASE.$CMP.RESTART_HISTORY_STREAM.DATE.nc
│ $CASE.$CMP.RESTART_STREAM.DATE.nc
│ rpointer.$MODEL
│ ...
The st_archive tool essentially reorganizes the output in the run dir:
$CASE> tree
.
├── archive
│ ...
│ ├── $CMP
│ ├── hist
│ ├── ...
│ ├── $CASE.$CMP.HISTORY_STREAM.DATE.nc
│ ├── ...
│ ├── logs
│ ├── rest
│ ...
│ ├── $DATE
│ ├── rpointer.$CMP
│ ├── $CASE.$CMP.RESTART_HISTORY_STREAM.$DATE.nc
│ ├── $CASE.$CMP.RESTART_STREAM.$DATE.nc
├── build
│ ...
├── case_scripts
│ ...
│ ├── case.st_archive
│ ├── CaseStatus
│ ├── case.submit
│ ...
├── run
│ ...
│ $CASE.$CMP.HISTORY_STREAM.LAST_DATE.nc
│ $CASE.$CMP.RESTART_HISTORY_STREAM.LAST_DATE.nc
│ $CASE.$CMP.RESTART_STREAM.LAST_DATE.nc
│ rpointer.$MODEL
│ ...
Proposal: instead of writing all sorts of random things into the run dir, we should directly output and organize files into an archive dir with the following enhancement:
- buildnml handles creating
- archive/$CMP/history
- archive/$CMP/restart_history
- archive/$CMP/restart
- archive/logs
- model automatically saves into corresponding directories during runtime
- case.st_archive can be relegated to creating symlinks between existing files and a new "rpointer_restarts" dir that will correspond to the old archive/rest/0000-00-00-00000 dirs
Feedback & discussion welcome.