issue-tracking icon indicating copy to clipboard operation
issue-tracking copied to clipboard

Rerun + archive experiments from REST API

Open neighthan opened this issue 5 years ago • 2 comments

(I've discussed this a bit on Slack, but I thought I'd make an official issue so that I can see when support for this comes; these are important features for me)

I often have several experiments fail for related reasons. I find it easier to analyze these failures in Python because I can easily aggregate by the error messages (possibly allowing for small differences, e.g. in whitespace). Once I resolve whatever errors caused the failures, I would like to be able to re-run the failed experiments and archive the previous run so that only the new (hopefully fixed) runs will show up. I would imagine this being implemented as an archive method on APIExperiment which does the same as clicking the archive button on the webpage for that experiment and, for allowing rerunning experiments, APIExperiment should have access to the information from the Reproduce section on the webpage (for this use case, only the Run command is needed, because one doesn't want to use the exact same code, but adding in the Reproduce section as well would be good since that should be easy to do at the same time and others could probably use that). This would improve my workflow a lot when I have hundreds of experiments in a project and need to sift through the errors to ensure I have all the results I need.

neighthan avatar Apr 02 '19 15:04 neighthan

You can now Archive an experiment from the REST/Python API using this method:

import comet_ml
comet_api = comet_ml.API()
comet_api.archive_experiment("some key")` 
comet_api.archive_experiments(["key1","key2"])` 

Still working on the reproduce data

gidim avatar Apr 05 '19 19:04 gidim

@neighthan note that you'll need to upgrade your comet_ml to 1.0.50.

dsblank avatar Apr 05 '19 20:04 dsblank

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Nov 22 '23 21:11 github-actions[bot]