biome-text
biome-text copied to clipboard
Invalid metric warning in HPO
Is your feature request related to a problem? Please describe. When specifying a metric in an HPO run with ray tune, we could try to check if that metric exists and can be used. Maybe is out of our hand, as this is passed to tune.run(), but biome is running in the background so maybe that metric can be checcked
Describe the solution you'd like Imagine:
analysis_frozen = tune.run( tune_exp, scheduler=tune.schedulers.ASHAScheduler(), metric="totally_fake_metric", mode="max", progress_reporter=tune.JupyterNotebookReporter(overwrite=True), )
We could print a warning/error.
This is just a reminder for me: The head should store the logged metric names + the information if shown in the pbar or not. This would make a quick check of the specified monitored metric very easy.