Onboarding: Support `metrics`, `params` and `plots` without `dvc.yaml`.

[daavoo]

DVC metrics, params and plots commands support --targets that allow to use the command without requiring dvc init / dvc.yaml.

For example, all the following renders as expected in a repo without .dvc nor dvc.yaml:

dvc metrics diff HEAD~1 --targets 'dvclive.json' && \
dvc params diff HEAD~1 --targets myparams.json && \
dvc plots diff HEAD~1 --targets 'dvclive' --open && 

If VSCode allows users to provide (and perhaps persist the values) these custom targets, it would enable a quick way of exploring the Experiment Table and Plots views without requiring to set up DVC (beyond being installed).

Studio already supports configuring custom metrics and parameters paths for a repo without .dvc/dvc.yaml

Regarding the table, https://github.com/iterative/vscode-dvc/issues/1966 would be a great added value for this use case

[dberenbaum]

If VSCode allows users to provide (and perhaps persist the values) these custom targets, it would enable a quick way of exploring the Experiment Table and Plots views without requiring to set up DVC (beyond being installed).

If we persist, why not do so in dvc.yaml?

Studio already supports configuring custom metrics and parameters paths for a repo without .dvc/dvc.yaml

In Studio, we would have to ask users to open a PR/merge changes to dvc.yaml to persist the configuration. In VS Code, I think this should be easier since it is working on the filesystem.

By the way, do we know whether anyone is using custom metrics and parameters in Studio?

[shcheklein]

By the way, do we know whether anyone is using custom metrics and parameters in Studio?

@tapadipti do we have these numbers?

If we persist, why not do so in dvc.yaml?

I also think we can generate dvc.yaml tbh. It makes it more important though to have global metrics / plots sections w/o pipelines. This way it we could do this automatically via context menu.

[tapadipti]

do we have these numbers?

There are about 100 repos with custom metrics or params files. Several, but not all, of these are Iterative repos.

@Suor I queried the repos_repo table and checked the field custom. Let me know if there's another table/field also to check.

[shcheklein]

It's not clear how to do this yet, tbh. I also tend to think that we'll be persisting this in dvc.yaml. We would need to support top level metrics, params then first. @dberenbaum do we have a plan for that?

[dberenbaum]

The latest proposal on this is in https://www.notion.so/iterative/Python-Notebook-Experiments-5a92a6f1677841bfb3613983f75a60b2. It looks likely that we will be prioritizing top-level metrics and params for this reason.

[shcheklein]

Let's for now postpone this, we've done the first step with - extensions now is trying to initialize and install DVC and suggests DVCLive.

Next step can be - add buttons to add metrics, params, data, etc in the table I think via global sections in DVC yaml.