gto icon indicating copy to clipboard operation
gto copied to clipboard

Workflow for getting artifacts in the first release

Open aguschin opened this issue 3 years ago • 5 comments

Following up on @dberenbaum feedback

Can I retrieve artifacts via GTO, or is that left to other tools? What is the expected workflow? For example, if I do gto which rf production and get back v1.0.0, what do I do next if I want to use that artifact?

Until https://github.com/iterative/gto/issues/80 is not done, we need to establish some kind of workflow for that.

aguschin avatar Mar 23 '22 09:03 aguschin

There are two parts to it:

  1. Getting non-virtual artifacts from the registry. This should be pretty trivial after https://github.com/iterative/gto/issues/25
  2. Getting virtual artifacts. To make this workflow happen, we need some command that will give desired git reference given artifact version or env. Something like
$ gto latest rf --ref
abc1234

$ gto latest rf --path
models/random-forest.pkl

# getting artifacts that are currently promoted to envs
$ gto which rf production --ref
$ gto which rf production --path

# getting specific versions of artifacts
$ gto version rf v1 --ref
$ gto version rf v1 --path

This will enable the following usage

$ dvc get $REPO $(gto which rf production --path) --ref $(gto which rf production --ref)

Some advanced commands would include also:

# getting the known version
$ gto version rf v1 --ref

# getting the latest at some point in time
$ gto latest rf --ref --as-of "2022-03-25 10:15"

# getting the what was in env at some point in time
$ gto which rf production --ref --as-of "2022-03-25 10:15"

aguschin avatar Mar 25 '22 07:03 aguschin

Yes, listing the ref and path for now would at least create some viable workflow for how users can access the artifact (git clone && git checkout ref && ...).

dberenbaum avatar Mar 25 '22 14:03 dberenbaum

(Q: is there a separate ticket for the Python API for this?)

shcheklein avatar Mar 25 '22 15:03 shcheklein

(Q: is there a separate ticket for the Python API for this?)

No, but both CLI and API will be implemented as part of this ticket. Right now there is a parity between CLI and API, so each command in CLI has API counterpart and vise versa.

aguschin avatar Mar 28 '22 05:03 aguschin

Implemented part of this in #96

aguschin avatar Mar 28 '22 15:03 aguschin