pagmo2 icon indicating copy to clipboard operation
pagmo2 copied to clipboard

More MO optimization performance metrics

Open Sceki opened this issue 4 years ago • 2 comments

It would be useful to have some more performance metrics for MO optimization.

For instance, some that are often used in MO papers are:

  • [ ] Diversity of the non-dominated front
  • [ ] Closeness to the true Pareto-optimal front

These two can used only when the true Pareto optimal front is known (as it is the case for many test-suites such as zdt, dtlz, etc.). The aforementioned two metrics are described in: Deb, K.: Multi-Objective Optimization using Evolutionary Algorithms, John Wiley & Sons, Chichester, UK (2001).

Sceki avatar Sep 19 '19 15:09 Sceki

These metrics are a bit outdated, but I would propose to implement some of the quality indicators from:

Coello Coello and Falcon-Cardona,: Convergence and Diversity Analysis of Indicator-based Multi-Objective Evolutionary Algorithms, in GECCO 2019.

CoolRunning avatar Sep 19 '19 15:09 CoolRunning

These metrics are a bit outdated, but I would propose to implement some of the quality indicators from: Coello Coello and Falcon-Cardona,: Convergence and Diversity Analysis of Indicator-based Multi-Objective Evolutionary Algorithms, in GECCO 2019.

Yes, those metrics are Pareto-compliant. Please don't add metrics that are not Pareto-compliant. Proper performance assessment for MO optimization is explained in these works:

  • https://iridia-ulb.github.io/references/#ZitThiLauFon2003:tec
  • https://iridia-ulb.github.io/references/#KnoThiZit06:tutorial
  • https://iridia-ulb.github.io/references/#ZitKnoThi2008quality

Notable advances in metrics since those papers were published are:

  • Averaged Hausdorff distance: https://iridia-ulb.github.io/references/#SchEsqLarCoe2012tec
  • IGD+ : https://iridia-ulb.github.io/references/#IshMasTanNoj2015igd
  • Weighted hypervolume: https://iridia-ulb.github.io/references/#AugBadBroZit2009gecco

You can find implementations (in C) of IGD+ and Average Hausdorff in https://mlopez-ibanez.github.io/eaf/

MLopez-Ibanez avatar Jun 16 '20 23:06 MLopez-Ibanez