dvclive
dvclive copied to clipboard
iterative
📈 Log and track ML metrics, parameters, models with Git and/or DVC
When using `dvclive` alongside `DVC checkpoints`, each `step` is associated with a checkpoint hash. Afaik, the relation between `step` and `checkpoint` is currently only available in the `dvc exp show`...
I think it could be useful to expose something similar to the parsing utilities used in tests (i.e. [`read_logs`](https://github.com/iterative/dvclive/blob/master/tests/test_main.py#L21)). This could enable an easy way of parsing `dvclive` outputs into...
Users may want to save metrics every epoch but only save models every few epochs. Can we support this in dvclive?
It could be convenient, not sure how much, to let the user configure a remap from the `name`s passed to `dvclive.log` to some given `target`s. Internally, could be easily implemented....
By default the values for each metric at the latest step are being saved in the `summary`. It might be interesting to add an option to additionally save the **best**...
From [Optuna](https://github.com/optuna/optuna): > Optuna is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, define-by-run style user API. Thanks to our define-by-run API, the...
`log_model`
Use [MLEM](https://mlem.ai/) to save models. Should also enable these to be automatically tracked with DVC. This is important to be able to save models in DVCLive across frameworks in a...
Follow up to #322. The difference in behavior between `log_image` and `log_sklearn_plot` bothers me: ``` ├── plots │ ├── images │ │ ├── 0 │ │ │ └── img.png │...
"no step" / "step" logic was initially introduced to support different logging formats between step and not step updates. For `live.log_image`, "step" mode now overwrites the path instead of creating...
Not sure why anyone would want a longer method, but `log_metric` is more explicit. Is it worth adding for clarification?