test failures and warnings

[haakon-e]

Running tests on main lead to many warnings and some failures:

julia> include("test/runtests.jl")
┌ Warning: x_tol is deprecated. Use x_abstol or x_reltol instead. The provided value (1.0e-5) will be used as x_abstol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:110
┌ Warning: f_tol is deprecated. Use f_abstol or f_reltol instead. The provided value (1.0e-5) will be used as f_reltol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:120
┌ Warning: x_tol is deprecated. Use x_abstol or x_reltol instead. The provided value (1.0e-5) will be used as x_abstol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:110
┌ Warning: f_tol is deprecated. Use f_abstol or f_reltol instead. The provided value (1.0e-5) will be used as f_reltol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:120
┌ Warning: x_tol is deprecated. Use x_abstol or x_reltol instead. The provided value (1.0e-5) will be used as x_abstol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:110
┌ Warning: f_tol is deprecated. Use f_abstol or f_reltol instead. The provided value (1.0e-5) will be used as f_reltol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:120
┌ Warning: x_tol is deprecated. Use x_abstol or x_reltol instead. The provided value (1.0e-5) will be used as x_abstol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:110
┌ Warning: f_tol is deprecated. Use f_abstol or f_reltol instead. The provided value (1.0e-5) will be used as f_reltol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:120
Calibrate by Ensemble Kalman Processes and Get results: Test Failed at /Users/haakon/Documents/CliMA/clima-codes/KinematicDriver.jl/test/unit_tests/calibration_pipeline/test_optimization_utils.jl:26
  Expression: norm((u.ϕ_optim .- u_true) ./ u_true) < 0.1
   Evaluated: 0.12760951942246132 < 0.1

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:679 [inlined]
 [2] macro expansion
   @ ~/Documents/CliMA/clima-codes/KinematicDriver.jl/test/unit_tests/calibration_pipeline/test_optimization_utils.jl:26 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
 [4] top-level scope
   @ ~/Documents/CliMA/clima-codes/KinematicDriver.jl/test/unit_tests/calibration_pipeline/test_optimization_utils.jl:3
Calibrate by Ensemble Kalman Processes and Get results: Test Failed at /Users/haakon/Documents/CliMA/clima-codes/KinematicDriver.jl/test/unit_tests/calibration_pipeline/test_optimization_utils.jl:42
  Expression: norm((u.ϕ_optim .- u_true) ./ u_true) < 0.13
   Evaluated: 3.977800248755642 < 0.13

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:679 [inlined]
 [2] macro expansion
   @ ~/Documents/CliMA/clima-codes/KinematicDriver.jl/test/unit_tests/calibration_pipeline/test_optimization_utils.jl:42 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
 [4] top-level scope
   @ ~/Documents/CliMA/clima-codes/KinematicDriver.jl/test/unit_tests/calibration_pipeline/test_optimization_utils.jl:3
┌ Warning: x_tol is deprecated. Use x_abstol or x_reltol instead. The provided value (1.0e-5) will be used as x_abstol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:110
┌ Warning: f_tol is deprecated. Use f_abstol or f_reltol instead. The provided value (1.0e-5) will be used as f_reltol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:120
┌ Warning: x_tol is deprecated. Use x_abstol or x_reltol instead. The provided value (1.0e-5) will be used as x_abstol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:110
┌ Warning: f_tol is deprecated. Use f_abstol or f_reltol instead. The provided value (1.0e-5) will be used as f_reltol.
└ @ Optim ~/.julia/packages/Optim/8dE7C/src/types.jl:120
Test Summary:                                            | Pass  Fail  Total     Time
Unit Tests                                               |  358     2    360  1m18.9s
  Precipitation types                                    |   14           14     0.6s
  Moisture type                                          |    7            7     0.0s
  Initialise state                                       |   48           48     0.1s
  Tendency helper functions                              |  109          109     7.2s
  Parameter overwrites                                   |    3            3     0.0s
  Make space                                             |    1            1     0.2s
  Make rhs function                                      |    1            1     0.0s
  Initialise aux                                         |   11           11     0.7s
  advection_tendency                                     |    4            4     3.0s
  advection_tendency p3                                  |    6            6     2.4s
  aerosol activation for 2M schemes                      |    3            3     0.5s
  Make space                                             |    1            1     1.5s
  Make rhs function                                      |    1            1     0.0s
  Initialise aux                                         |    6            6     2.1s
  advection tendency                                     |    4            4     6.3s
  Get validation parameters                              |    4            4     2.6s
  Get validation samples                                 |    4            4     9.0s
  filter field                                           |    6            6     0.5s
  Get observational data                                 |   12           12     2.8s
  Get Observations                                       |    4            4     0.7s
  creating parameters object                             |    5            5     0.0s
  Run KiD                                                |    5            5     2.1s
  Run KiD (multiple cases) and run dynamical model       |    4            4     0.4s
  Test PCA computations                                  |    6            6     0.5s
  Constructing reference statistics                      |   30           30     1.3s
  normalize observations                                 |    2            2     0.3s
  Constructing priors                                    |    3            3     0.0s
  Initial parameter ensemble                             |    2            2     0.2s
  Calibrate by Ensemble Kalman Processes and Get results |   11     2     13    11.6s
  EKP with mini-batching                                 |    2            2     0.7s
  Calibrate by Optim and Get results                     |    5            5     5.2s
  Calibrate function for abstract optimization style     |    1            1     0.1s
  Compute loss                                           |    2            2     0.2s
  Find limits                                            |    4            4     0.3s
  make filter                                            |    4            4     0.1s
  get numbers from config                                |    3            3     0.0s
  get single case data from vector                       |                 0     0.1s
  make block diagonal matrix                             |    5            5     0.1s
  compute error metrics                                  |    3            3     0.5s
  lwp rwp and rainrate                                   |    6            6     0.5s
  make output directory - save - load                    |    5            5     2.8s
  plot correlation map                                   |    1            1     2.1s
ERROR: LoadError: Some tests did not pass: 358 passed, 2 failed, 0 errored, 0 broken.
in expression starting at /Users/haakon/Documents/CliMA/clima-codes/KinematicDriver.jl/test/runtests.jl:3

[haakon-e]

cc: @trontrytel

[trontrytel]

All the failures are related to the calibration pipeline. I.e. all are coming from this file: calibration_pipeline/run_calibration_pipeline_unit_tests.jl Sorry, but I didn't have the bandwidth to maintain that. The model itself is working fine.

Test Summary: | Pass  Total  Time
Unit Tests    |  219    219  1.0s
[ Info: Precompiling JETReviseExt [f03d6c7f-2be2-5ab6-8acb-706271f84fa0] (cache misses: wrong dep version loaded (2))
mass_thresholds = (5.0e-10, Inf)
Test Summary:      | Pass  Total   Time
Optimization Tests |   10     10  19.6s
Test Summary:         | Pass  Total  Time
Initial Profile Tests |   12     12  2.7s
Test Summary: | Total  Time
Box Test      |     0  5.8s
[ Info: ("EquilibriumMoisture", "Precipitation1M", "CliMA_1M", "CliMA_1M")
[ Info: Initialising
[ Info: Solving
[ Info: Plotting
[ Info: Saved animation to /Users/ajaruga_clima/clones/KinematicDriver.jl/test/experiments/KiD_driver/Output_EquilibriumMoisture_Precipitation1M_CliMA_1M/figures/animation.mp4
Test Summary:   | Total     Time
K1D Driver Test |     0  4m41.2s
[ Info: ("CloudyPrecip", "CliMA_1M", "CliMA_1M")
[ Info: Initialising
dist_names = ("gamma", "gamma")
mass_thresholds = (5.0e-10, Inf)
[ Info: Solving
[ Info: Plotting
Test Summary:                 | Total     Time
Collision Sedimentation Tests |     0  1m21.8s
[ Info: Saved animation to /Users/ajaruga_clima/clones/KinematicDriver.jl/test/experiments/Ki2D_driver/Output_NonEquilibriumMoisture_Precipitation1M_CliMA_1M/rhoqtot.mp4
[ Info: Saved animation to /Users/ajaruga_clima/clones/KinematicDriver.jl/test/experiments/Ki2D_driver/Output_NonEquilibriumMoisture_Precipitation1M_CliMA_1M/rhoqliq.mp4
[ Info: Saved animation to /Users/ajaruga_clima/clones/KinematicDriver.jl/test/experiments/Ki2D_driver/Output_NonEquilibriumMoisture_Precipitation1M_CliMA_1M/rhoqrai.mp4
Test Summary:        | Total     Time
K2D Simulation Tests |     0  6m59.3s