iterative
ref: formalize `dvc.yaml` page + better structure
In general, https://dvc.org/doc/user-guide/project-structure/dvcyaml-files is turning into a reference closer to the cmd ref than guides, esp. since there are actual guides for certain topics in there (like pipelining or plots). But the content is still mixed with some guide-like sections and other more formal spec/definition sections. Should be closer to the latter.
Maybe also move it into /doc/files-reference along with a few other pages?
Some specific ideas from the OP:
...we should be describing things in their order. In every section we can put a small snippet again to get some context back what part of dvc.yaml we are talking about.
Section names could be a bit more formal e.g. "cmd: stage command"
Originally posted by @shcheklein in https://github.com/iterative/dvc.org/pull/3414#discussion_r954298890
https://dvc.org/doc/user-guide/project-structure/dvcyaml-files is focused on pipelines. With coming changes to the experiments onboarding experience that will introduce top-level params and metrics in addition to the existing plots, it's time to broaden the scope of how we describe dvc.yaml.
It can be a general configuration file for ML experiments. This enables users to:
- Describe experiment tracking info: pipelines, parameters, metrics, and plots.
- Configure that info in human-readable files that can be modified before, during, or after experiments.
- Manage the configuration alongside code and data in Git.
- Use DVCLive to configure it, but also have flexibility to manually configure additional outputs (like adding plots that aren't supported by DVCLive).
- Use the tool of your choice to consume this schema and visualize/compare experiments (CLI, VS Code, Studio, CML, IPython/notebooks in the future).
