NagDFOLS its actually good for normal optimization

[inikishev]

I just tried by making my function return a size 1 array and it works really well, it actually beats cobyla on a 2d deformable coregistration task

[janosg]

Hi @inikishev, thanks for sharing your experience. Feel free to add a code snippet so we can have a look.

The paper where DFO-LS is developed clearly states that it's goal is to solve least-squares problems. The LS in the name is also for least-squares.

However, in optimization anything is possible. So it is very possible that in your particular example DFO-LS works well even though you are not solving an actual least-squares problem. In general, however, I would expect optimizers like nlopt_bobyqa, nlopt_newuoa or any of the nelder mead implementations to perform better on such problems.