guide: updates to and around `dvc.yaml` doc

[jorgeorpinel]

Extracted from https://github.com/iterative/dvc.org/pull/3414#issuecomment-1189898267

This page should focus on the schema/spec of the file and it's sections more than guidance and explaining concepts, since we'll have a guide for that now (see https://github.com/iterative/dvc.org/pull/3414).

[github-actions[bot]]

Link Check Report

• content/docs/command-reference/config.md

  • PASS: /doc/user-guide/project-structure/dvcyaml-files#dictionary-unpacking = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/project-structure/dvcyaml-files#dictionary-unpacking (200)
  • PASS: /doc/user-guide/project-structure/dvcyaml-files#dictionary-unpacking = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/project-structure/dvcyaml-files#dictionary-unpacking (200)

• content/docs/command-reference/dag.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)

• content/docs/command-reference/exp/init.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#stage-commands = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#stage-commands (200)

• content/docs/command-reference/move.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)

• content/docs/command-reference/repro.md

  • PASS: https://www.gnu.org/software/make/manual/ (200)
  • PASS: /doc/command-reference/dag = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/command-reference/dag (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#avoiding-unexpected-behavior = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#avoiding-unexpected-behavior (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#stage-commands = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#stage-commands (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)

• content/docs/command-reference/run.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#avoiding-unexpected-behavior = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#avoiding-unexpected-behavior (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#stage-commands = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#stage-commands (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)
  • PASS: /docs/user-guide/how-to/add-deps-or-outs-to-a-stage = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/docs/user-guide/how-to/add-deps-or-outs-to-a-stage (200)

• content/docs/command-reference/stage/add.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#stage-commands = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#stage-commands (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#dvcyaml-metafiles = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#dvcyaml-metafiles (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)
  • PASS: /docs/user-guide/how-to/add-deps-or-outs-to-a-stage = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/docs/user-guide/how-to/add-deps-or-outs-to-a-stage (200)

• content/docs/command-reference/stage/index.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#stages = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#stages (200)

• content/docs/start/data-management/metrics-parameters-plots.md

  • PASS: /doc/start/data-pipelines#dependency-graphs = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/start/data-pipelines#dependency-graphs (200)

• content/docs/start/data-management/pipelines.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)

• content/docs/user-guide/basic-concepts/pipeline.md

  • PASS: /doc/user-guide/project-structure/dvcyaml-files = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/project-structure/dvcyaml-files (200)

• content/docs/user-guide/experiment-management/running-experiments.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)

• content/docs/user-guide/machine-learning-pipelines/defining-pipelines.md

  • PASS: https://en.wikipedia.org/wiki/Deterministic_algorithm (200)
  • PASS: https://en.wikipedia.org/wiki/Software_entropy (200)
  • PASS: https://www.gnu.org/software/software.html (200)
  • PASS: /doc/user-guide/external-dependencies = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/external-dependencies (200)
  • PASS: /doc/user-guide/project-structure/dvcyaml-files#stage-entries = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/project-structure/dvcyaml-files#stage-entries (200)
  • PASS: https://en.wikipedia.org/wiki/Directed_acyclic_graph (200)
  • PASS: /doc/start/data-management = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/start/data-management (200)
  • PASS: /doc/user-guide/project-structure/pipelines-files#templating = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/project-structure/pipelines-files#templating (200)
  • PASS: /doc/user-guide/project-structure/internal-files#structure-of-the-cache-directory = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/project-structure/internal-files#structure-of-the-cache-directory (200)
  • PASS: https://medium.com/all-things-ai/in-depth-parameter-tuning-for-random-forest-d67bb7e920d (Excluded)

• content/docs/user-guide/machine-learning-pipelines/index.md

  • PASS: /doc/start/data-management/pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/start/data-management/pipelines (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)

• content/docs/user-guide/project-structure/dvcyaml-files.md

  • PASS: /doc/command-reference/run = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/command-reference/run (200)
  • PASS: https://yaml.org/ (200)
  • PASS: https://www.gnu.org/software/software.html (200)
  • PASS: /doc/command-reference/metrics = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/command-reference/metrics (200)
  • PASS: /doc/command-reference/plots = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/command-reference/plots (200)
  • PASS: /doc/user-guide/how-to/merge-conflicts#dvcyaml = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/how-to/merge-conflicts#dvcyaml (200)
  • PASS: /doc/user-guide/project-structure/dvc-files#output-entries = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/project-structure/dvc-files#output-entries (200)
  • PASS: /doc/user-guide/experiment-management/checkpoints = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/experiment-management/checkpoints (200)
  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines#parameter-dependencies = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines#parameter-dependencies (200)
  • PASS: /doc/command-reference/status#local-workspace-status = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/command-reference/status#local-workspace-status (200)
  • PASS: https://github.com/iterative/dvcyaml-schema (200)
  • PASS: /doc/install/plugins#visual-studio-code = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/install/plugins#visual-studio-code (200)
  • PASS: /doc/install/plugins#pycharmintellij = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/install/plugins#pycharmintellij (200)

• content/docs/user-guide/related-technologies.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)

• content/docs/user-guide/what-is-dvc.md

  • PASS: /doc/user-guide/machine-learning-pipelines/defining-pipelines = https://dvc-org-guide-dvc-yaml-rx4eyee.herokuapp.com/doc/user-guide/machine-learning-pipelines/defining-pipelines (200)

All 50 links passed!

[shcheklein]

@jorgeorpinel what is the status for this?

[jorgeorpinel]

Ready for review @shcheklein but should probably not be merged before #3414, whence it came.

#3414 is also ready for review but it's too big again so I need to split it more and didn't have time yet, sorry.

[dberenbaum]

@jorgeorpinel What's the status of this PR? Is it still needed?

[jorgeorpinel]

I'm not sure how this PR got so complicated 😓 but yes, after mergingmain (resolved a bunch of conflicts) I see many useful changes here. I will have to split this though, otherwise it's too hard to review... ⏳

[jorgeorpinel]

It's 88 commits because this was started in July. I later removed a bunch of changes.

Agree about the relevance of these changes. At that time I already had a bunch of edits stashed for this file which I didn't manage to propose since they never seemed major enough so eventually I just made the PR (and then it grew out of control).

But don't worry, in the new docs process, this kind of work is not something I'll be paying much attention to in the foreseeable future.

[github-actions[bot]]

548749a

Link Check Report

There were no links to check!