issue-tracking icon indicating copy to clipboard operation
issue-tracking copied to clipboard

Incorrect bounds in optimizer config will lead to unclear error message for random and bayes algorithm

Open kjappelbaum opened this issue 4 years ago • 2 comments

Using comet-ml 3.1.7 on OSX. The following snippet, with some fit method that returns a score

config = {
        "algorithm": "bayes",
        "parameters": {
            "learning_rate": {
                "type": "float",
                "min": 0.00,
                "max": 0.3,
                "scalingType": "loguniform",
            }}
}
opt = Optimizer(config, api_key=os.environ["COMET_API_KEY"])

for i, experiment in enumerate(opt.get_experiments()):
        params = {
            "learning_rate": experiment.get_parameter("learning_rate"),
        }
        loss = fit(experiment, X_train, y_train, params)
        experiment.log_metric("loss", loss)

"Fails" with algorithm=bayes without error message, just printing

COMET INFO: Optimizer search 83050ee9015f4c1db1cbe883934dda31 has completed

With algorithm=random it raises an error, but on the first look it seems unrelated to the real issue

requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: https://optimizer.comet.ml/create

Only with algorithm=grid the error message shows what the real problem is

comet_ml.exceptions.OptimizerException: Optimizer server: Range exceeds valid bound

Using a non-zero bound the snippets works fine for all algorithms, but I guess it would be nice if the error would be caught for all algorithms and maybe even point out more clearly what the issue is (e.g, it should be non-zero).

kjappelbaum avatar May 19 '20 10:05 kjappelbaum

Thanks @kjappelbaum for the detailed report. We are now investigating...

dsblank avatar May 19 '20 11:05 dsblank

We'll work towards making all of the errors bubble up to be messages for the client. Thanks for the report!

dsblank avatar May 21 '20 18:05 dsblank

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Nov 09 '23 21:11 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Nov 14 '23 21:11 github-actions[bot]