DFO-LS is a flexible package for solving nonlinear least-squares minimization, without requiring derivatives of the objective. It is particularly useful when evaluations of the objective function are expensive and/or noisy. DFO-LS is more flexible version of DFO-GN.
The main algorithm is described in our paper [1] below.
If you are interested in solving general optimization problems (without a least-squares structure), you may wish to try Py-BOBYQA, which has many of the same features as DFO-LS.
See manual.pdf or here.
The development of DFO-LS is outlined over several publications:
- C Cartis, J Fiala, B Marteau and L Roberts, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, ACM Transactions on Mathematical Software, 45:3 (2019), pp. 32:1-32:41 [preprint arXiv 1804.00154] .
- M Hough and L Roberts, Model-Based Derivative-Free Methods for Convex-Constrained Optimization, SIAM Journal on Optimization, 21:4 (2022), pp. 2552-2579 [preprint arXiv 2111.05443].
- Y Liu, K H Lam and L Roberts, Black-box Optimization Algorithms for Regularized Least-squares Problems, arXiv preprint arXiv:arXiv:2407.14915, 2024.
If you use DFO-LS in a paper, please cite [1]. If your problem has constraints, including bound constraints, please cite [1,2]. If your problem includes a regularizer, please cite [1,3].
DFO-LS requires the following software to be installed:
- Python 3.9 or higher (http://www.python.org/)
Additionally, the following python packages should be installed (these will be installed automatically if using pip, see Installation using pip):
- NumPy (http://www.numpy.org/)
- SciPy version 1.11 or higher (http://www.scipy.org/)
- Pandas (http://pandas.pydata.org/)
Optional package: DFO-LS versions 1.2 and higher also support the trustregion package for fast trust-region subproblem solutions. To install this, make sure you have a Fortran compiler (e.g. gfortran) and NumPy installed, then run pip install trustregion. You do not have to have trustregion installed for DFO-LS to work, and it is not installed by default.
DFO-LS can be directly installed in Anaconda environments using conda-forge:
$ conda install -c conda-forge dfo-lsFor easy installation, use pip as root:
$ pip install DFO-LSNote that if an older install of DFO-LS is present on your system you can use:
$ pip install --upgrade DFO-LSto upgrade DFO-LS to the latest version.
Alternatively, you can download the source code from Github and unpack as follows:
$ git clone https://github.com/numericalalgorithmsgroup/dfols $ cd dfols
DFO-LS is written in pure Python and requires no compilation. It can be installed using:
$ pip install .
To upgrade DFO-LS to the latest version, navigate to the top-level directory (i.e. the one containing pyproject.toml) and rerun the installation using pip, as above:
$ git pull $ pip install .
If you installed DFO-LS manually, you can test your installation using the pytest package:
$ pip install pytest $ python -m pytest --pyargs dfols
Alternatively, the HTML documentation provides some simple examples of how to run DFO-LS.
Examples of how to run DFO-LS are given in the documentation, and the examples directory in Github.
If DFO-LS was installed using pip you can uninstall as follows:
$ pip uninstall DFO-LS
If DFO-LS was installed manually you have to remove the installed files by hand (located in your python site-packages directory).
Please report any bugs using GitHub's issue tracker.
This algorithm is released under the GNU GPL license. Please contact NAG for alternative licensing.