Gradient-Free-Optimizers
Gradient-Free-Optimizers copied to clipboard
SimonBlanke
Spiral Optimization
In the upcoming v1.1 I will release the Spiral Optimization algorithm. An explanation and visualization can be found on wikipedia.
The current source code of this new algorithm for Gradient-Free-Optimizers can be found on the v1.1 branch.
There are still some things to implement/unterstand:
- At the moment the algorithm has almost no parameters. So I have to put variables into the hard-coded values of some parameter, that make sense.
- The parameter
r(k)in the equation of the wikipedia-page is described as the step-rate, but it behaves move like a radius of the spiral movement. Maybe it makes sense to lower this radius over time to converge to the current best position? - What should the algorithm do if convergence is reached? This is also a current problem of PSO, but solving this problem does not seem to be the part of the standard versions of those two algorithms. Maybe there are modifications that solve this problem.
Depending on the start configuration of positions the particles might stick to the edges of the search-space when attempting to spiral:
This looks much better, but there is nothing happening after convergence:
I added some fixes for the rotation-matrix and added a decay-parameter to change the step-size (or more like: radius-factor) over time. In the following search-path-gif you can see what happens, when the decay-rate is >1. The particles start to spiral out after some time. Very interesting!
