FATAL: MPP_DOMAINS_DEFINE.INC: NOT ALL THE PE_END ARE IN THE PELIST

[mmesnie]

Hello,

Is there a configuration file and machine file that are known to work with the ubuntu20.04-gnu9.3-ufs-srwapp container?

I’ve tried modifying ush/config.community.sh and ush/machine/singularity.sh for my local environment, but continue receiving the below error when I run run_fcst.sh. All of the previous stages complete successfully.

Thanks in advance for any help.

-Mike

(regional_workflow) root@ip-172-31-0-67:/opt/ufs-srweather-app/regional_workflow/ush/wrappers# time ./run_fcst.sh

…

FATAL: mpp_domains_define.inc: not all the pe_end are in the pelist

…

---

MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD

with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.

You may or may not see output from other processes, depending on

exactly when Open MPI kills them.

---

ERROR:

From script: "exregional_run_fcst.sh"

Full path to script: "/opt/ufs-srweather-app/regional_workflow/scripts/exregional_run_fcst.sh"

           Call to executable to run FV3-LAM forecast returned with nonzero exit

code.

Exiting with nonzero status.

ERROR:

From script: "JREGIONAL_RUN_FCST"

Full path to script: "/opt/ufs-srweather-app/regional_workflow/jobs/JREGIONAL_RUN_FCST"

Call to ex-script corresponding to J-job "JREGIONAL_RUN_FCST" failed.

Exiting with nonzero status.

[mmesnie]

In case others experience this, the FATAL error below was due to the VM not having enough CPU cores. Increasing from 8 to 16 cores solved the problem.

FATAL: mpp_domains_define.inc: not all the pe_end are in the pelist