DeepForest icon indicating copy to clipboard operation
DeepForest copied to clipboard

More model registry to cometML?

Open bw4sz opened this issue 2 years ago • 2 comments

https://www.comet.com/site/products/machine-learning-model-versioning/

Comet has a robust set of model registry tools. That allow versioning and archiving. This would be useful if we start to gather more models and versions of models. Currently there isn't alot of iteration in the baseline models, so it might be overkill at the current model. Clearly the current workflow of stashing the model in the git release and checking a local .csv about what model we have used is pretty hacky and bound to break eventually.

Thoughts @ethanwhite?

bw4sz avatar Feb 13 '23 17:02 bw4sz

I definitely think we should move to formal model registry as part of our overall process. Let's just make sure to set it up on the group workspace so that we don't have releases for the same general models in different places depending on who ran that model version. In general I think putting all experiments in the group workspace is better practice for easier collaboration anyway, so if anything this should encourage other good habits.

ethanwhite avatar Feb 16 '23 15:02 ethanwhite

An added benefit here is that the github releases and the pypi releases can match. Its odd that they do not match currently. Currently we check the release when loading a baseline model.

>>> m.use_release()
Model from DeepForest release https://github.com/weecology/DeepForest/releases/tag/1.0.0 was already downloaded. Loading model from file.
Loading pre-built model: https://github.com/weecology/DeepForest/releases/tag/1.0.0

bw4sz avatar Apr 11 '23 01:04 bw4sz