Mateusz Baran

There is also this: https://arxiv.org/abs/1811.00980

I mostly meant the array power representation of power manifold but maybe Optimization.jl would accept Jacobians represented by nested arrays, I will have to check.

I see, I think I will skip such constraints in OptimizationManopt.jl for the first version.

There is a Julia library that implements the Euclidean variant: https://github.com/ImperialCollegeLondon/DirectSearch.jl (MIT license). It could potentially be helpful.

The closest thing I could find is move forcing: https://arxiv.org/pdf/1906.08867.pdf . Do you have any literature that uses the current stopping criterion?

OK, then it's not any better in this regard :wink:

It's effectively very similar in the idea, very easy to implement and much faster, I don't see any drawbacks. Standard stopping criteria for PSO are usually more simple, see https://web2.qatar.cmu.edu/~gdicaro/15382/additional/CompIntelligence-Engelbrecht-ch16.pdf...

Decreasing step size works relatively well in my problem so for me it is more a nice-to-have thing for comparison than something I need, so it can wait.

Not at the moment but it looks like a relatively simple modification of RLM to support it. The rescaling described in http://ceres-solver.org/nnls_modeling.html#theory looks like it would work for RLM too.

@kellertuer do you think it would be better to add robustification support to the current implementation of RLM or make a separate implementation?