ncvarsort.py does not seem to preserve order

[glemieux]

Describe the issue

While attempting to use UpdateParamAPI.py for the API37.1 updates, I found that the order of the cdl file was getting changed. The tool appears to run correctly based on the command line output, but diffing the output shows that the default parameter file has the variable order changed.

I ended up cloning another version of fates in a separate directory, checking out sci.1.80.4_api.37.0.0, which just prior to the last tooling update. I had to do some slight path modifications to the tooling in that repo, but calling the older tag tooling from the original fates directory resulted in the correct ordering.

Relevant log output

# packages in environment at /home/glemieux/local/miniconda3/envs/fates-tools:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
blosc                     1.21.6               hef167b5_0    conda-forge
bzip2                     1.0.8                h4bc722e_7    conda-forge
c-ares                    1.34.2               heb4867d_0    conda-forge
ca-certificates           2024.8.30            hbcca054_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cftime                    1.6.4           py312hc0a28a1_1    conda-forge
esmf                      8.7.0           nompi_h6063b07_1    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
hdf4                      4.2.15               h2a13503_7    conda-forge
hdf5                      1.14.4          nompi_h2d575fe_101    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
ld_impl_linux-64          2.43                 h712a8e2_2    conda-forge
libaec                    1.1.3                h59595ed_0    conda-forge
libblas                   3.9.0           20_linux64_openblas    conda-forge
libcblas                  3.9.0           20_linux64_openblas    conda-forge
libcurl                   8.10.1               hbbe4b11_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libexpat                  2.6.4                h5888daf_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.2.0               h77fa898_1    conda-forge
libgcc-ng                 14.2.0               h69a702a_1    conda-forge
libgfortran               14.2.0               h69a702a_1    conda-forge
libgfortran-ng            14.2.0               h69a702a_1    conda-forge
libgfortran5              14.2.0               hd5240d6_1    conda-forge
libgomp                   14.2.0               h77fa898_1    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           20_linux64_openblas    conda-forge
libnetcdf                 4.9.2           nompi_h2564987_115    conda-forge
libnghttp2                1.64.0               h161d5f1_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.25          pthreads_h413a1c8_0    conda-forge
libsqlite                 3.47.0               hadc24fc_1    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx                 14.2.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.2.0               h4852527_1    conda-forge
libudunits2               2.2.28               h40f5838_3    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.13.4               h064dc61_2    conda-forge
libzip                    1.11.1               hf83b1b0_0    conda-forge
libzlib                   1.3.1                hb9d3cd8_2    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
nco                       5.2.9                hd7e902f_1    conda-forge
ncurses                   6.5                  he02047a_1    conda-forge
netcdf-fortran            4.6.1           nompi_ha5d1325_108    conda-forge
netcdf4                   1.7.2           nompi_py312ha728dd9_100    conda-forge
numpy                     1.26.4          py312heda63a1_0    conda-forge
openssl                   3.4.0                hb9d3cd8_0    conda-forge
pip                       24.3.1             pyh8b19718_0    conda-forge
python                    3.12.7          hc5c86c4_0_cpython    conda-forge
python_abi                3.12                    5_cp312    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scipy                     1.11.4          py312heda63a1_0    conda-forge
setuptools                75.3.0             pyhd8ed1ab_0    conda-forge
snappy                    1.2.1                ha2e4443_0    conda-forge
tempest-remap             2.2.0                h1819be2_5    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.3.1                hb9d3cd8_2    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge

FATES tag

sci.1.80.14_api.37.0.0

Host land model tag

ctsm5.3.024

Machine

other HLM supported machine

Other supported machine name

lobata

[glemieux]

On a related note, I think troubleshooting this issue would have been slightly improved by making the tooling directory it's own repo and adding it as a submodule to the fates repo. This would have allowed me to checkout older versions of the tools within the existing directory structure.