Update SFS test to C192mx025; add debug and restart tests; update ultralow warmstart input to specify V2 file type.

[DeniseWorthen]

Commit Queue Requirements:

• Fill out all sections of this template.
• All sub component pull requests have been reviewed by their code managers.
• Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
• Commit 'test_changes.list' from previous step

---

Description:

1. Replaces current C96mx100 SFS test with a configuration matching the current SFS C192mx025 prototype configuration. Adds a restart and debug test for SFS. The SFS input data is from CORe.

Warmstarts for ICE and OCN are from

/gpfs/f6/sfs-emc/scratch/Neil.Barton/ICs/CPC/C192mx025/sfs.19940430/18/mem000/model

The ATM ICs are from

/gpfs/f6/sfs-emc/scratch/Neil.Barton/ICs/CPC/C192mx025/sfs.19940501/00/mem000/model/atmos/input

The ATM and OCN increment files are from

/gpfs/f6/sfs-emc/scratch/Neil.Barton/ICs/CPC/C192mx025/sfs.19940501/00/mem001/analysis

The fix files are identical to those in

/gpfs/f6/drsa-precip3/world-shared/role.glopara/fix/orog/20240917/C192
/gpfs/f6/drsa-precip3/world-shared/role.glopara/fix/ugwd/20240624/C192

For example, the oro files (e.g. oro_data.tile1.nc) used by the RT are source from

FV3_fix_tiled/C192mx025/oro_C192.mx025.tile1.nc

and these are identical to those in role.glopara/fix/orog/20240917/C192/C192.mx025_oro_data.tile2.nc

However, the C192 grid files (eg C192_grid.tile4.nc) are not identical . In the RTs, these are source from

FV3_input_data192/INPUT/C192_grid.tile1.nc

and these files have small round-off level differences w/ the grid files in role.glopara/fix/orog/20240917/C192/. I don't believe this is an issue unless an attempt is made to repro a G-W run and an RT run.

Note that because the ATM is writing history files every 24 hours for the control test, the restart and debug tests compare only restart files from all components; the surface history files will not be B4B because the accumulated fields in the surface files will not be B4B.

At commit 3acafc1, this is the input.nml difference between the new cpld_control_sfs and the initial SFS sandbox provided. I've removed the differences in the fix files, which differ only the path-to-file.

diff -yw --suppress-common-lines /gpfs/f6/infra-cpu/world-shared/Denise.Worthen/sfs192mx025.rundir/input.nml cpld_control_sfs_intel/input.nml | grep -v FN

sandbox	rt test
res_latlon_dynamics = atminc.nc	res_latlon_dynamics = 'atminc.nc'
	mraerosol = .false.
	lthailaware = .false.
	do_ngw_ec = .true.
	use_oceanuv = .false.
STOCHINI=.false.,	STOCHINI=.false.
	new_lscale=.true.
skebint = 1800	skebint = 0
shumint = 3600	shumint = 0
spptint = 1800	spptint = 0
LNDP_TAU=21600	LNDP_TAU=21600,
LNDP_LSCALE=500000	LNDP_LSCALE=500000,
output_directory = 'MOM6_OUTPUT/',	output_directory = './MOM6_OUTPUT',
restart_output_dir = 'MOM6_RESTART/'	restart_output_dir = './RESTART/'

2. Replaces the current ultra-low warmstart staged inputs with files properly containing the V2 global attribute. The warmstart files were created using this branch at 569d4ac

Commit Message:

* UFSWM - Point RTs to new input-data directory with fixes for ultralow warmstart files and new SFS configuration
* UFSWM - Update configuration of SFS test to C192mx025 and add a restart and debug test
* UFSWM - Rename diag table template for SFS to diag_table_sfs.IN
* UFSWM - Add namelist parameter to control use of ECMWF non-stationary gravity wave scheme
* UFSWM - Add namelist parameter to set new_lscale for stochastic physics
  * FV3 - Add 'file_version' global attribute to sfc_data restart files, as defined in input sfc_data files
  * stochastic_physics - Restrict print messages to be on root PE only

Priority:

• Normal

Git Tracking

UFSWM:

• Fixes Create C192mx025 regression test configuration #2764
• Fixes diagnostic messages are written on all PEs and not controlled by flag to turn off NOAA-PSL/stochastic_physics#87
• Fixes Restart outputs are not b4b identical to cold-start run when sfc v2 files are used #2827
• Fixes Add namelist parameter to enable ECMWF non-stationary gravity wave scheme #2840
• Fixes Ultralow warmstart/restart tests do not properly test V2 file type #2917
• Fixes Add new_lscale as a configurable parameter #2923

Sub component Pull Requests:

• FV3: Add 'file_version' global attribute to sfc_data restart files NOAA-EMC/ufsatm#997
• stochastic_physics: print messages on rootpe only NOAA-PSL/stochastic_physics#88

UFSWM Blocking Dependencies:

• None

Documentation:

• No documentation update is required for this PR (please explain). No non-documented features are added.

---

Changes

Regression Test Changes (Please commit test_changes.list):

Baseline changes are due to

1. Changes for tests relying on V2 global attribute in the ATM restart files
2. Changes for test configuration from c96mx100->c192mx025 for SFS

Pre-testing on C6 using a staged baseline for the ultralow and SFS test shows that:

1. Warmstart and restart tests for ultralow and new SFS tests reproduce
2. SFS runs and reproduces in debug mode

Input data Changes:

• New input data.

This PR requires a NEW input-data directory to both add the files required for the new SFS test as well as fixing the ultralow staged input.

The NEW input-data directory is located on Ursa at

/scratch4/NCEPDEV/nems/Denise.Worthen/input-data-2025XXXX

Library Changes/Upgrades:

• No Updates

---

Testing Log:

• RDHPCS
  • Hera
  • Orion
  • Hercules
  • GaeaC6
  • Derecho
  • Ursa
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
• opnReqTest (complete task if unnecessary)

[XiaqiongZhou-NOAA]

I was reproducing what Neil gave me as a working SFS run directory. The current test here reproduces those settings.

@NeilBarton-NOAA can you comment on whether these values need to be updated?

These should be included in SFS RT since SFS are also using SPPTINT=SKEBINT=SHUMINT=0 as GEFS.

[DeniseWorthen]

Thanks @XiaqiongZhou-NOAA ! And the new_lscale = .true. also, right?

[XiaqiongZhou-NOAA]

SFS is using default new_lscale=.false, but I guess it is better to be consistent with GEFS. Please also include it as true. Thanks.

[NeilBarton-NOAA]

note, the SPPT configuration is currently being tunned and the values will change in the future.

[DeniseWorthen]

@NeilBarton-NOAA Thanks. It is a moving target! I'd like to reproduce whatever is the current SFS "base" configuration for this initial commit. As you zero in on parameter sets etc, we can update the test.

[DeniseWorthen]

@NickSzapiro-NOAA The only new_lscale settings I see are in regional input.nml templates. There is no parsable variable for the global_control.nml.IN and the code sets it .false. by default. You refer to GEFS, but this must be a setting they've hardwired for the G-W runs because it is not how we're testing in the RTs.

@NeilBarton-NOAA @XiaqiongZhou-NOAA The RT can only really test the features that have been tested and are known/expected to be included in the SFS configuration. As new features and parameterizations/values are tested and confirmed in the science experiments, the test will update and also implement those features etc. But if these new stoch settings are still under evaluation in the SFS science side, then I'm not sure we want to update them in the test.

[NickSzapiro-NOAA]

We've haven't updated GEFS RT yet, including change of new_lscale=.true.. The code makes it seem that's more "proper" and should be used. I don't know if stochastic physics side has preference, when they can

[DeniseWorthen]

@gspetro-NOAA Is the project PR board being updated automatically yet? This PR is still shown as draft and that the pre-tests are not done. I've committed both the ursa log and the test change list and this PR has been open for the last 5 days.

The issue in UFS-UTILS has been confirmed by others as non-impactful.

[NeilBarton-NOAA]

We have tested new_lscale=.true.. in GEFS. For SFS, the default is to test with new_lscale=.true..

[gspetro-NOAA]

@DeniseWorthen The status update will likely have to be manual, but I have automated the labeling for baselines/input data based on labels we add to the PR. I am hoping to automate the priority labels and info about subcomponents involved, but that will have to wait till after my current work on the Runtime/Memory reporting/scorecard. I will bring this PR up at the WM CM meeting tomorrow to make sure we get the input data staged and this PR scheduled. I think the main holdup on the input data is just that we need to confer and agree on what date to use before we can stage it.

[DeniseWorthen]

We have tested new_lscale=.true.. in GEFS. For SFS, the default is to test with new_lscale=.true..

Thanks @NeilBarton-NOAA I will add an issue to create a parsable variable and add it to the global nml template for use.

[gspetro-NOAA]

@DeniseWorthen We will go with input-data-20251015, although I suspect your PR may be merged a bit after that date. I will work on transferring the data tomorrow (Thursday).

[gspetro-NOAA]

@DeniseWorthen I have rsync'd the new input data to:

/scratch4/NAGAPE/epic/role-epic/UFS-WM_RT/NEMSfv3gfs/input-data-20251015

I had to resolve some role account issues, hence the delay, but I'm hoping it should work as expected now! Let me know if there are any issues.

[DeniseWorthen]

@gspetro-NOAA There is an issue w/ missing links in the new input data

/scratch4/NAGAPE/epic/role-epic/UFS-WM_RT/NEMSfv3gfs/input-data-20251015/FV3_fire_input_data/cameron_peak_fire/INPUT

In the staged input, you'll see a set of files as links. These links need to be restored in the official input-data directory:

/scratch4/NCEPDEV/nems/Denise.Worthen/input-data-2025XXXX/FV3_fire_input_data/cameron_peak_fire/INPUT: ls -l
total 791796
-rw-r--r-- 1 Denise.Worthen nems  10017137 Oct  5 17:41 C3336_grid.tile7.halo3.nc
lrwxrwxrwx 1 Denise.Worthen nems        31 Oct  5 19:14 gfs_bndy.tile7.000.nc -> srw.t18z.gfs_bndy.tile7.f000.nc
lrwxrwxrwx 1 Denise.Worthen nems        31 Oct  5 19:14 gfs_bndy.tile7.003.nc -> srw.t18z.gfs_bndy.tile7.f003.nc
lrwxrwxrwx 1 Denise.Worthen nems        20 Oct  5 19:14 gfs_ctrl.nc -> srw.t18z.gfs_ctrl.nc
lrwxrwxrwx 1 Denise.Worthen nems        32 Oct  5 19:15 gfs_data.nc -> srw.t18z.gfs_data.tile7.halo0.nc
-rw-r--r-- 1 Denise.Worthen nems  10207409 Oct  5 17:41 grid.tile7.halo4.nc
-rw-r--r-- 1 Denise.Worthen nems      9352 Oct  5 17:42 grid_spec.nc
-rw-r--r-- 1 Denise.Worthen nems   3379751 Oct  5 17:41 oro_data.nc
-rw-r--r-- 1 Denise.Worthen nems   3647271 Oct  5 17:41 oro_data.tile7.halo4.nc
-rw-r--r-- 1 Denise.Worthen nems   2017664 Oct  5 17:41 oro_data_ls.nc
-rw-r--r-- 1 Denise.Worthen nems   2017700 Oct  5 17:41 oro_data_ss.nc
lrwxrwxrwx 1 Denise.Worthen nems        32 Oct  5 19:14 sfc_data.nc -> srw.t18z.sfc_data.tile7.halo0.nc
-rw-r--r-- 1 Denise.Worthen nems  61473531 Oct  5 17:41 srw.t18z.gfs_bndy.tile7.f000.nc
-rw-r--r-- 1 Denise.Worthen nems  61473531 Oct  5 17:41 srw.t18z.gfs_bndy.tile7.f003.nc
-rw-r--r-- 1 Denise.Worthen nems      9252 Oct  5 17:42 srw.t18z.gfs_ctrl.nc
-rw-r--r-- 1 Denise.Worthen nems 230908335 Oct  5 17:41 srw.t18z.gfs_data.tile7.halo0.nc
-rw-r--r-- 1 Denise.Worthen nems  20194253 Oct  5 17:41 srw.t18z.sfc_data.tile7.halo0.nc

We also need to be sure the sym-links are maintained when you stage the new input-data on other platforms. But lets get Ursa fixed first.

[gspetro-NOAA]

@DeniseWorthen I think I've added the symlinks properly now, but let me know if I missed anything.

[DeniseWorthen]

@gspetro-NOAA The input-data is now correct. The test list change is as expected (v2 tests, ultralow and sfs). The conus13km_debug_intel test timed out and has been removed from the test change list.

This PR is ready once the input-data has been added to all platforms. Thanks.

[gspetro-NOAA]

Great! I will go ahead and add the input data onto other platforms! Then we can hopefully get this into the Commit Queue asap!