Loosen the restriction in RTs that Mediator runs on <300 Tasks

[DeniseWorthen]

Description

The tasking for CMEPS has always been limited to using less than 300 tasks. If memory serves, this arose from the time when Moorthi was running high-resolution coupled cases and he would find that the runs would hang if CMEPS was given high PE counts.

While I cannot be 100% certain, I believe this behaviour was due to the use of serial-netcdf in CMEPS (reading/writing restarts and mesh files for the marine components), rather than using PIO/pnetcdf configuration.

We currently now use PIO/pnetcdf for CMEPS, and this restriction can be at least loosened. In recent testing for the operational GFSv17 configuration for example, CMEPS was set at 4800 ranks (and 4 threads).

For the new coupled C1152 configuration for the weekly RTs, the restriction on 300 needs to be relaxed.

Solution

Modify these lines in rt_utils.sh

ufs-weather-model/tests/rt_utils.sh

Lines 96 to 98 in 4bee751

	# MED - mediator (CMEPS) runs on at most 300 tasks.
	MED_compute_tasks=$((ATM_compute_tasks<=300 ? ATM_compute_tasks : 300))
	med_petlist_bounds="0 $((MED_compute_tasks - 1))"

I would suggest replacing 300 with 1200.

Alternatives

Related to

[DeniseWorthen]

@DusanJovic-NOAA I did try manually setting MED_compute_tasks in the new cpld_c1152 test, but it was still over-written by the limit in rt_utils.sh. Is there any way to set the compute tasks in the test itself?

[DusanJovic-NOAA]

@DusanJovic-NOAA I did try manually setting MED_compute_tasks in the new cpld_c1152 test, but it was still over-written by the limit in rt_utils.sh. Is there any way to set the compute tasks in the test itself?

Currently, no. The MED_compute_tasks is computed, as you showed in the link in the issue description. If we want to make it a configurable parameter, then the above line where it is computed should be removed.

[DeniseWorthen]

OK, so we could either just make it == to the ATM compute tasks or just loosen it to 1200. I think that solution is sufficient for the RTs; G-W might want to revisit how they also limit it but I'm sure they don't use rt_utils.sh.