Porting the SciPy tests to Fortran

[ivan-pi]

One of the suggestions in #14 (comment) was to port the MINPACK tests from SciPy to our test suite.

I've tried to locate some of the relevant files:

• https://github.com/scipy/scipy/blob/820fe86bf3ad8ba1c5647632b131659ed7bd447e/scipy/optimize/tests/test_nonlin.py (nice to see @certik among the authors)
• https://github.com/scipy/scipy/blob/820fe86bf3ad8ba1c5647632b131659ed7bd447e/scipy/optimize/tests/test_minpack.py
• https://github.com/scipy/scipy/blob/820fe86bf3ad8ba1c5647632b131659ed7bd447e/scipy/optimize/tests/test_least_squares.py
• https://github.com/scipy/scipy/blob/820fe86bf3ad8ba1c5647632b131659ed7bd447e/scipy/optimize/tests/test__root.py

Note that some of the test may only be testing properties of the SciPy interface, and not have much to do with Fortran at all.

[ilayn]

@mdhaber Is there anything we can provide for numerical stability and edge cases for MINPACK specifically?

[mdhaber]

Hmm, I don't know of any. Like stress tests for the underlying algorithms?

[ilayn]

Yes mostly testing the edge cases and maybe stress testing that we are sure it doesn't break at the Fortran side

[ivan-pi]

By edge cases, do you mean for examples things like NaN or infinity in the initial guess? I believe that MINPACK currently doesn't perform such checks. The question is whether to add such input-checking to the Fortran code, or leave it as the responsibility of the caller (or SciPy wrapper) to perform such checks.

A good place to start is maybe to look at the errors that MINPACK detects currently, e.g. in hybrd:

        integer, intent(out) :: info             !! an integer output variable. if the user has
                                            !! terminated execution, `info` is set to the (negative)
                                            !! value of `iflag`. see description of `fcn`. otherwise,
                                            !! `info` is set as follows:
                                            !!
                                            !!  * ***info = 0*** improper input parameters.
                                            !!  * ***info = 1*** relative error between two consecutive iterates
                                            !!    is at most `xtol`.
                                            !!  * ***info = 2*** number of calls to `fcn` has reached or exceeded
                                            !!    `maxfev`.
                                            !!  * ***info = 3*** `xtol` is too small. no further improvement in
                                            !!    the approximate solution `x` is possible.
                                            !!  * ***info = 4*** iteration is not making good progress, as
                                            !!    measured by the improvement from the last
                                            !!    five jacobian evaluations.
                                            !!  * ***info = 5*** iteration is not making good progress, as
                                            !!    measured by the improvement from the last
                                            !!    ten iterations.

and compare with the wrappers in SciPy (https://github.com/scipy/scipy/blob/main/scipy/optimize/_minpack_py.py) and figure out what could be simplified by handling in Fortran.

[ilayn]

Historically, we have been quite successful with finding wrapper bugs. What I had in mind is more like ill-conditioned problems, singularities, getting stuck in while loops and so on. But I don't have concrete examples yet.

[ivan-pi]

There are a number of test collections including ill-conditioned problems. We have issues open for a few (#10, #35) of them. Here are a few original references:

• An Evaluation of Mathematical Software that Solves Nonlinear Least Squares Problems (1981) | ACM TOMS
• Testing Unconstrained Optimization Software (1981) | ACM TOMS (the original MINPACK tests)
• A New Nonlinear Equations Test Problem (1985) | Rice.edu
• The MINPACK-2 test problem collection (1992) | OSTI.GOV
• NIST Statistical Reference Dataset - Nonlinear Regression (2003) | NIST (I'm staging these in Fortran here, but haven't had time to finish them yet)

It's just a lot of effort to create the drivers in Fortran (or Python), and integrate them with the CI/CD in a way that we could track the correctness and speed improvements with time.

[ilayn]

Is there any way that I can make life easier for you folks? CI/CD setup is something we can help you with, since we are doing this very often, though I think we don't have much Fortran knowledge.

Just copy pasting numbers from horrific PDFs to text files is also fine, any way you need help please let us know.