[Feature] Support contour_plot to show the parameter relationship.

[c-bata]

Feature Request

Porting plot_contour() function to TypeScript. https://github.com/optuna/optuna/blob/v2.5.0/optuna/visualization/_contour.py

[2403hwaseer]

Hi @c-bata , I'd like to work on this issue.

[c-bata]

Cool! I assigned you 👍

[2403hwaseer]

Hi @c-bata ! I wanted to ask if it is okay to add two select boxes so that the user can choose the parameters for which the graph needs to be shown. Showing the contour plot of all parameters at once will make the graph very crowded.

[c-bata]

Hi @c-bata ! I wanted to ask if it is okay to add two select boxes so that the user can choose the parameters for which the graph needs to be shown. Showing the contour plot of all parameters at once will make the graph very crowded.

Sounds good!

[c-bata]

This issue is once introduced by @2403hwaseer but reverted due to the problems on dynamic search space like the following:

study = optuna.create_study(
    study_name="single-dynamic", storage=storage, direction="maximize"
)

def objective_single_dynamic(trial: optuna.Trial) -> float:
    category = trial.suggest_categorical("category", ["foo", "bar"])
    if category == "foo":
        return (trial.suggest_float("x1", 0, 10) - 2) ** 2
    else:
        return -((trial.suggest_float("x2", -10, 0) + 5) ** 2)

study.optimize(objective_single_dynamic, n_trials=50)

study = optuna.create_study(
    study_name="multi-dynamic", storage=storage, directions=["minimize", "minimize"]
)

def objective_multi_dynamic(trial: optuna.Trial) -> Tuple[float, float]:
    category = trial.suggest_categorical("category", ["foo", "bar"])
    if category == "foo":
        x = trial.suggest_float("x1", 0, 5)
        y = trial.suggest_float("y1", 0, 3)
        v0 = 4 * x ** 2 + 4 * y ** 2
        v1 = (x - 5) ** 2 + (y - 5) ** 2
        return v0, v1
    else:
        x = trial.suggest_float("x2", 0, 5)
        y = trial.suggest_float("y2", 0, 3)
        v0 = 2 * x ** 2 + 2 * y ** 2
        v1 = (x - 2) ** 2 + (y - 3) ** 2
        return v0, v1

study.optimize(objective_multi_dynamic, n_trials=50)

Please see https://github.com/optuna/optuna-dashboard/pull/79 for more details.

[2403hwaseer]

@c-bata I did not consider the above mentioned case while writing the code for the issue. Can you please help me understand what dynamic search spaces are?

[c-bata]

As the source code which I commented, different parameters may be sampled on each trials. For example, please suppose that category="foo" is sampled on the first trial in the following objective function, then x1 parameter is sampled but x2 parameter isn't. But if category="bar" is sampled on the next trial, it does not contains x1 parameter.

def objective_single_dynamic(trial: optuna.Trial) -> float:
    category = trial.suggest_categorical("category", ["foo", "bar"])
    if category == "foo":
        x1 = trial.suggest_float("x1", 0, 10)
        return (x1 - 2) ** 2
    else:
        x2 = (trial.suggest_float("x2", -10, 0)
        return -(x2 + 5) ** 2)

This is the dynamic search space I mean. I added a test case for dynamic search spaces in visual_regression_test.py. So please check the output of the following command.

$ pip install -r requirements.txt
$ python visual_regression_test.py --output-dir tmp  # screenshots will be saved in "tmp/" directory

https://github.com/optuna/optuna-dashboard/blob/main/DEVELOPMENT.md#running-visual-regression-tests-using-pyppeteer

[2403hwaseer]

@c-bata Thanks for the detailed explanation! I'll figure this out and work on making the implementation better this time.

[c-bata]

Resolved at https://github.com/optuna/optuna-dashboard/pull/287