Consider other performance metrics than the pareto front plot for multi-objective benchmark

[HideakiImamura]

Motivation

Recently, we have introduced the multi-objective benchmark suite in Optuna. The current performance metric is only the pareto font plot. It is difficult to judge the performance from the picture. We would like to know other quantitative performance metrics.

The current pareto front plot looks like this.

Description

The example of other performance metrics are as follows. See the survey paper for more details.

• The number of trials v.s. the achieved best hypervolume.
• The number of trials v.s. the achieved best delta index.
• The number of trials v.s. the achieved best number of points on the pareto front.

These are examples, so we may adopt other metrics. We have the following requirements for our performance metrics.

• Each metric should be the function whose input is the sequence of trials and output is one dimensional float value.
• Each metric should be plotted in a similar format to the optimization history.

Alternatives (optional)

No response

Additional context (optional)

No response

[github-actions[bot]]

This issue has not seen any recent activity.

[github-actions[bot]]

This issue has not seen any recent activity.

[github-actions[bot]]

This issue was closed automatically because it had not seen any recent activity. If you want to discuss it, you can reopen it freely.

[not522]

Ref: #4748 and related PRs introduced plot_hypervolume_history.