dvclive icon indicating copy to clipboard operation
dvclive copied to clipboard

`dvc`: Enable model loading/saving based on `checkpoint: true`

Open daavoo opened this issue 4 years ago • 6 comments

Given a DVC stage where checkpoints are enabled:

stages:
  train:
    cmd: src/train.py
    deps:
    - data
    - src
    outs:
    - model.pth:
        checkpoint: true
    live:
      dvclive:
        summary: true
        html: true

Users need to also remember enabling checkpoints on the stage code.

Maybe we could treat the model_file argument of the integrations similar to how we treat the path argument:

A) If checkpoint: true is set in DVC and model_file is None in DVCLive: we use the value of the output as model_file . B) If checkpoint: true is set in DVC and model_file is set in DVCLive but does not match: raise ConfigMismatchError. C) If checkpoint: false, just use whichever value has model_file in DVCLive.

daavoo avatar Nov 12 '21 15:11 daavoo

@pared @dberenbaum

daavoo avatar Nov 12 '21 15:11 daavoo

A) If checkpoint: true is set in DVC and model_file is None in DVCLive: we use the value of the output as model_file .

What do you mean by "we use the value of the output as model_file"? Does that mean infer model.pth here?

dberenbaum avatar Nov 12 '21 19:11 dberenbaum

A) If checkpoint: true is set in DVC and model_file is None in DVCLive: we use the value of the output as model_file .

What do you mean by "we use the value of the output as model_file"? Does that mean infer model.pth here?

Yes. Make DVC store the output path in an env var, like it does for DVCLIVE_PATH

daavoo avatar Nov 12 '21 21:11 daavoo

@daavoo That would work only in integrations, right?

pared avatar Nov 15 '21 11:11 pared

@daavoo That would work only in integrations, right?

With the current implementation, yes.

daavoo avatar Nov 15 '21 14:11 daavoo

It makes sense to me.

It still seems slightly awkward to have to define the path in dvc.yaml. It would be great if this wasn't needed at all, and the model file was part of the live output by default, but that may be a separate issue.

dberenbaum avatar Nov 15 '21 15:11 dberenbaum