dvclive icon indicating copy to clipboard operation
dvclive copied to clipboard

remote/no-git logging

Open dberenbaum opened this issue 1 year ago • 7 comments

There are several scenarios where it's not realistic to work in a git repo:

  • Databricks
  • Google colab
  • SageMaker training jobs

People may run notebooks or jobs in these environments that are associated with a Git repo, but access to the git repo is awkward when running that notebook or job. In this case, dvclive+studio can still provide a typical experiment tracking experience (logging metrics, plots, etc. to the studio server). If the studio token and project are available, dvclive should still be able to log to studio to see live updates and associate them with an experiment.

It can be out of scope for initial implementation, but it would also be useful to persist the metrics and plots logged in this scenario back to git. If they are not tracked in git, they will be lost when that experiment is merged, reproduced, etc. It might be possible to reverse engineer the metrics and plots from studio and add them back to the git repo.

dberenbaum avatar Jul 25 '23 11:07 dberenbaum