Update figures/gifs on optimizer history

[hmgaudecker]

For the envisioned screencasts, we need black backgrounds and single files instead of the gifs.

This got me started, but I am not quite happy with the looks yet. So we should do the move to plotly for this bit as part of this PR.

What needs improvement IMO:

• [ ] The two evaluation points on the x-axis
  • [ ] make sense for tracking the entire history of actual optimizers (although it would be cool to make them more bold and fade out as a function of the distance of the current evaluation (say, i-10 and later get opacity 20% and current evaluation 100%, equal steps in between)
  • [ ] are not salient enough for the stylized algorithms.
• [ ] Some visualization of trust region would be great, where applicable. Maybe just shade the entire background?
• [ ] For gifs, initial evaluation should get a separate screen without the step to the left (Nelder-Mead), right now it is not fully obvious what is meant
• [ ] Always differentiate between current iteration and next step (different markers), add both to function. Currently looks weird for gradient-based optimizers. Markers always on top of function and gradient.
• [ ] Add evaluation points to gradient-free trust region.
• [ ] Add counter of the number of iterations or other visualization for actual algorithms, zoom in, or the like -- currently nothing whatsoever is visible during the the second half of, e.g., COBYLA or Nelder-Mead

[codecov[bot]]

Codecov Report

Merging #273 (8a6bb22) into main (6b4576d) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #273   +/-   ##
=======================================
  Coverage   90.85%   90.85%           
=======================================
  Files         171      171           
  Lines       12439    12439           
=======================================
  Hits        11302    11302           
  Misses       1137     1137           

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 6b4576d...8a6bb22. Read the comment docs.

[janosg]

The code here can probably be improved by using the history collection introduced in #324

[janosg]

We close this due to a more ambitious visualization projects started in #403