keepsake
keepsake copied to clipboard
replicate
Version control for machine learning
The convenience function `CheckpointList.metrics` require you to know the name of the metric you want to return. It might be better to just eagerly construct a dictionary of all metrics...
Currently: * you can try to plot non-numeric metrics, which fails with matplotlib error * you don't get warnings if metrics are missing in for any of the experiments in...
Tensorboard has a smoothing slider, that's useful when metrics have high variance between steps. We should add that to all our plotting functions in the Python API.
The validation of replicate.yaml in Python is currently very rudimentary. We should make it more robust and provide the user with better error messages.
# Why If you pass an `.ipynb` file to `path=...` in `init()` or `checkpoint()`, it will record an old version of the notebook, not the notebook at the point you...
You can do `checkpoint.open()` and `checkpoint.checkout()`, but not `experiment.open()` and `experiment.checkout()`. You should probably be able to do this. There is some design complexity here. Do we also check out...
# Why Sometimes you need to do more than just diff the code, training data, and metadata. Perhaps you want to surface particular metrics, or calculate the Jensen-Shannon divergence. Or...
# Why It would be useful to define how to run inferences, how to run training, etc in a standard format, so that tools can integrate with models stored in...
# Why When working in a notebook, a lot of work you do is not training, and it is very easy to lose your work in a notebook. It should...
# Why Replicate is used a lot in notebooks, and it would be nice if there was a way to integrate better with the notebook environment. # What A Jupyter...