dvc icon indicating copy to clipboard operation
dvc copied to clipboard

dvc experiments checkout: option to select the best experiment regarding one metric

Open gcoter opened this issue 5 years ago • 2 comments

I had an idea while trying the new experiment feature. For the use case of grid-search or hyper-parameter optimization, could we imagine an option for dvc experiments checkout to checkout the best experiment regarding one of the metrics? Something like dvc experiments checkout --best metrics.json:accuracy. With an option like this, you can run (manually or automatically) a lot of experiments and ask dvc to return the best experiment, which can be then commited to git. Or instead of --best, it could be --lowest or --highest to take into account that some metrics should be minimized and some others maximized.

gcoter avatar Jul 25 '20 14:07 gcoter

Commenting on a workaround for reference, in case someone reaches this issue.

Behavior like the description can be achieved (in a UNIX system) as follows:

dvc exp apply $(dvc exp show --csv --sort-by metrics.json:accuracy | tail -n 1 | cut -d , -f 1)

daavoo avatar Apr 17 '23 12:04 daavoo

Related: #8775.

dberenbaum avatar Apr 17 '23 14:04 dberenbaum