Description
In what will be ctsm5.3.016 I have a list of tests that fail to build on Izumi today. That worked previous to today (so I wonder if system changes happened). However, ctsm5.3.014 with MPIserial_2.5.1, cime6.1.37 and ccs_config_cesm1.0.10 is building fine. So it's just the new tag that has cime6.1.49 and the same version of MPIserial and ccs_config.
Here's the complete list it includes: nag, intel, and gnu compilers:
ERS_D_Ld5_Mmpi-serial.1x1_vancouverCAN.I1PtClm50SpRs.izumi_nag.clm-CLM1PTStartDate.GC.ctsm5316acl_nag
ERS_D_Ld7_Mmpi-serial.1x1_smallvilleIA.IHistClm50BgcCropRs.izumi_intel.clm-decStart1851_noinitial.GC.ctsm5316acl_int
ERS_D_Mmpi-serial_Ld5.1x1_brazil.I2000Clm50FatesRs.izumi_nag.clm-FatesCold.GC.ctsm5316acl_nag
ERS_Ld1200_Mmpi-serial.1x1_numaIA.I2000Clm50BgcCropQianRs.izumi_gnu.clm-cropMonthlyNoinitial.GC.ctsm5316acl_gnu
ERS_Ld1600_Mmpi-serial.1x1_numaIA.I2000Clm50BgcCropQianRs.izumi_intel.clm-cropMonthlyNoinitial.GC.ctsm5316acl_int
ERS_Ld600_Mmpi-serial.1x1_smallvilleIA.I1850Clm50BgcCrop.izumi_gnu.clm-cropMonthlyNoinitial.GC.ctsm5316acl_gnu
ERS_Ly20_Mmpi-serial.1x1_numaIA.I2000Clm50BgcCropQianRs.izumi_intel.clm-cropMonthlyNoinitial.GC.ctsm5316acl_int
ERS_Ly20_Mmpi-serial.1x1_numaIA.I2000Clm50BgcCropQianRs.izumi_intel.clm-cropMonthlyNoinitial--clm-matrixcnOn.GC.ctsm5316acl_int
ERS_Ly3_Mmpi-serial.1x1_smallvilleIA.IHistClm50BgcCropQianRs.izumi_gnu.clm-cropMonthOutput.GC.ctsm5316acl_gnu
ERS_Ly5_Mmpi-serial.1x1_smallvilleIA.I1850Clm50BgcCrop.izumi_gnu.clm-ciso_monthly.GC.ctsm5316acl_gnu
ERS_Ly5_Mmpi-serial.1x1_smallvilleIA.I1850Clm50BgcCrop.izumi_gnu.clm-ciso_monthly--clm-matrixcnOn.GC.ctsm5316acl_gnu
ERS_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm50BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int
ERS_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm50BgcCropQianRs.izumi_intel.clm-cropMonthOutput--clm-matrixcnOn_ignore_warnings.GC.ctsm5316acl_int
SMS_D_Ld1_Mmpi-serial.f45_f45_mg37.I2000Clm50SpRs.izumi_gnu.clm-ptsRLA.GC.ctsm5316acl_gnu
SMS_D_Ld1_Mmpi-serial.f45_f45_mg37.I2000Clm50SpRs.izumi_gnu.clm-ptsROA.GC.ctsm5316acl_gnu
SMS_D_Ld1_Mmpi-serial.f45_f45_mg37.I2000Clm50SpRs.izumi_nag.clm-ptsRLA.GC.ctsm5316acl_nag
SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int
SMS_D_Mmpi-serial_Ld5.5x5_amazon.I2000Clm60FatesRs.izumi_nag.clm-FatesCold.GC.ctsm5316acl_nag
SMS_Ld10_D_Mmpi-serial.CLM_USRDAT.I1PtClm60SpRs.izumi_nag.clm-default--clm-NEON-TOOL.GC.ctsm5316acl_nag
SMS_Ld5_Mmpi-serial.1x1_brazil.IHistClm60Bgc.izumi_gnu.clm-mimics.GC.ctsm5316acl_gnu
SMS_Ly3_Mmpi-serial.1x1_numaIA.I2000Clm50BgcDvCropQianRs.izumi_gnu.clm-ignor_warn_cropMonthOutputColdStart.GC.ctsm5316acl_gnu
SMS_Ly5_Mmpi-serial.1x1_brazil.IHistClm50BgcQianRs.izumi_intel.clm-newton_krylov_spinup.GC.ctsm5316acl_int
SMS_Ly5_Mmpi-serial.1x1_smallvilleIA.IHistClm60BgcCropQianRs.izumi_gnu.clm-gregorian_cropMonthOutput.GC.ctsm5316acl_gnu
For the SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput test the mpi-serial.bldlog. has:
cat /scratch/cluster/erik/tests_ctsm5316acl/SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int/bld/mpi-serial.bldlog.241219-131622
mpi_serial_path is /fs/cgd/data0/erik/ctsm_worktree/quickfix/libraries/mpi-serial
gmake --output-sync -f /scratch/cluster/erik/tests_ctsm5316acl/SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int/Tools/Makefile -C /scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial CIME_MODEL=cesm SMP=FALSE CASEROOT="/scratch/cluster/erik/tests_ctsm5316acl/SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int" CASETOOLS="/scratch/cluster/erik/tests_ctsm5316acl/SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int/Tools" CIMEROOT="/fs/cgd/data0/erik/ctsm_worktree/quickfix/cime" SRCROOT="/fs/cgd/data0/erik/ctsm_worktree/quickfix" COMP_INTERFACE="nuopc" COMPILER="intel" DEBUG="TRUE" EXEROOT="/scratch/cluster/erik/tests_ctsm5316acl/SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int/bld" RUNDIR="/scratch/cluster/erik/tests_ctsm5316acl/SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int/run" INCROOT="/scratch/cluster/erik/tests_ctsm5316acl/SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int/bld/lib/include" LIBROOT="/scratch/cluster/erik/tests_ctsm5316acl/SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.izumi_intel.clm-cropMonthOutput.GC.ctsm5316acl_int/bld/lib" MACH="izumi" MPILIB="mpi-serial" NINST_VALUE="c1a1l1" OS="LINUX" PIO_VERSION=2 SHAREDLIBROOT="/scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int" BUILD_THREADED="FALSE" USE_ESMF_LIB="TRUE" USE_MOAB="FALSE" COMP_ATM="datm" COMP_ICE="sice" COMP_GLC="sglc" COMP_LND="clm" COMP_OCN="socn" COMP_ROF="srof" COMP_WAV="swav" USE_TRILINOS="FALSE" USE_ALBANY="FALSE" USE_PETSC="FALSE" COMP_NAME=mpi-serial /scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial/Makefile.conf
gmake: Entering directory '/scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial'
SHAREDLIBROOT |/scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int| SHAREDPATH |intel/mpi-serial/debug/nothreads|
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... icc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether icc accepts -g... yes
checking for icc option to accept ISO C89... none needed
checking whether icc understands -c and -o together... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of icc... gcc3
checking whether we are using the GNU Fortran compiler... no
checking whether ifort accepts -g... yes
checking for ranlib... ranlib
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to get verbose linking output from ifort... -v
checking for Fortran libraries of ifort... -L/usr/local/intel-cluster-20.0.1/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin -L/usr/lib/gcc/x86_64-redhat-linux/8/ -L/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/ -L/lib/../lib64 -L/lib/../lib64/ -L/usr/lib/../lib64 -L/usr/lib/../lib64/ -L/usr/lib/gcc/x86_64-redhat-linux/8/../../../ -L/lib64 -L/lib/ -L/usr/lib64 -L/usr/lib -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lpthread -ldl -lirc_s
checking for dummy main to link with Fortran libraries... none
checking for Fortran name-mangling scheme... lower case, underscore, no extra underscore
checking how to run the C preprocessor... icc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking size of long... 8
checking if Fortran compiler supports allow-mismatch flag... yes
checking if Fortran compiler supports mismatch_all flag... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating tests/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
gmake: Leaving directory '/scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial'
gmake --output-sync -f /scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial/Makefile -C /scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial -j 4 SRCDIR=/fs/cgd/data0/erik/ctsm_worktree/quickfix/libraries/mpi-serial
gmake all-recursive
gmake[1]: Entering directory '/scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial'
cd /fs/cgd/data0/erik/ctsm_worktree/quickfix/libraries/mpi-serial && /bin/sh /fs/cgd/data0/erik/ctsm_worktree/quickfix/libraries/mpi-serial/missing automake-1.16 --gnu
gmake[1]: Leaving directory '/scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial'
configure.ac:3: error: version mismatch. This is Automake 1.16.1,
configure.ac:3: but the definition used by this AM_INIT_AUTOMAKE
configure.ac:3: comes from Automake 1.16.5. You should recreate
configure.ac:3: aclocal.m4 with aclocal and run automake again.
WARNING: 'automake-1.16' is probably too old.
You should only need it if you modified 'Makefile.am' or
'configure.ac' or m4 files included by 'configure.ac'.
The 'automake' program is part of the GNU Automake package:
<https://www.gnu.org/software/automake>
It also requires GNU Autoconf, GNU m4 and Perl in order to run:
<https://www.gnu.org/software/autoconf>
<https://www.gnu.org/software/m4/>
<https://www.perl.org/>
gmake[1]: *** [Makefile:414: /fs/cgd/data0/erik/ctsm_worktree/quickfix/libraries/mpi-serial/Makefile.in] Error 1
gmake: *** [/scratch/cluster/erik/tests_ctsm5316acl/sharedlibroot.ctsm5316acl_int/intel/mpi-serial/debug/nothreads/mpi-serial/Makefile:407: all] Error 2
ERROR: configure.ac:3: error: version mismatch. This is Automake 1.16.1,
configure.ac:3: but the definition used by this AM_INIT_AUTOMAKE
configure.ac:3: comes from Automake 1.16.5. You should recreate
configure.ac:3: aclocal.m4 with aclocal and run automake again.
WARNING: 'automake-1.16' is probably too old.
You should only need it if you modified 'Makefile.am' or
'configure.ac' or m4 files included by 'configure.ac'.
The 'automake' program is part of the GNU Automake package:
<https://www.gnu.org/software/automake>
It also requires GNU Autoconf, GNU m4 and Perl in order to run:
<https://www.gnu.org/software/autoconf>
<https://www.gnu.org/software/m4/>
<https://www.perl.org/>
gmake[1]: *** [Makefile:414: /fs/cgd/data0/erik/ctsm_worktree/quickfix/libraries/mpi-serial/Makefile.in] Error 1