EMAworkbench icon indicating copy to clipboard operation
EMAworkbench copied to clipboard

Docs: Expand on HyperVolume minimum and maximum parameters

Open EwoutH opened this issue 2 years ago • 1 comments

Currently the docs and the tutorial don't dive very deep into the HyperVolume input parameters when used as a convergence metric. In the examples they're (on the surface) quite arbitraily defined, especially the maximums, sometimes being 1, 1.01 or 1.1, and sometimes being 2 or 3. While the docs currently describes what it does and outputs, it doesn't really describe how the input parameters change its behaviour.

The docs and tutorial could include the following things:

  • How the input parameters change the HyperVolume behaviour as an convergence metric
  • How you can manually (or experimentally) determine realistic minimum and maximum HyperVolume input values
  • When and how HyperVolume.from_outcomes() can be used (when expected values are defined, etc.)

The docs can be expanded at the HyperVolume class while the tutorial could be expanded at the Tracking convergence part of the Directed search tutorial.

EwoutH avatar Jun 14 '22 13:06 EwoutH

I was actually thinking of dropping hypervolume and simplifying some of the convergence tracking. Based on experience over the last few years, best practice seems to be to log the archive every n generations. You can then, in post apply whatever metrics including hyper volume to analyze these logged archives.

Most of these metrics are available within platypus-opt, so we could then offload this functionality to platypus-opt. In the documentation or examples, we might still cover how to use the saved archives and platypus-opt for a few example metrics and include links to relevant literature if you want to know more. However, I don't want to write a tutorial on convergence metrics as part of the docs of the workbench.

quaquel avatar Jun 16 '22 18:06 quaquel