Support YAML anchors in configs for `CLI`

[rusmux]

trafficstars

🚀 Feature request

Support YAML anchors in configs for CLI.

Motivation

Anchors reduce duplication of configuration values. For example (not necessarily like this):

x-common-args: &common_args
  a: 1
  b: 2
  c: 3

instances:
  - class_path: __main__.MyClass
    init_args:
      name: instance_1
      <<: *common_arg
  - class_path: __main__.MyClass
    init_args:
      name: instance_2
      <<: *common_arg
  ...

Pitch

YAML anchors are supported by CLI.

Alternatives

[mauvilsa]

Than you for proposing! Note that yaml anchors are supported. What isn't possible is to add keys with arbitrary names, like x-common-args, which are not part of the signature. For example the following should work fine:

instances:
  - class_path: __main__.MyClass
    init_args: &init_args
      name: instance_1
      a: 1
      b: 2
      c: 3
  - class_path: __main__.MyClass
    init_args:
      <<: *init_args
      name: instance_2

[rusmux]

Oh, cool, thank you! Would it be possible to make CLI ignore extra values in configs?

[mauvilsa]

Ignoring extra content in configs is dangerous because other mistakes wouldn't be noticed. The same reason why parse_known_args is not implemented. So I am not really inclined to do this.

But you could handle this yourself by implementing a custom loader. That is:

import yaml
from jsonargparse import set_loader

def custom_yaml_load(stream):
    data = yaml.safe_load(stream)
    if isinstance(data, dict):
        data.pop("x-common-args")
    return data

set_loader("yaml", custom_yaml_load)

Some notes about this:

• The loader is used to parse everything (command line values, subconfigs, env vars), so the pop-ed key must be truly unique, otherwise valid data could be removed.
• jsonargparse's default yaml loader handles a few cases that the default yaml doesn't, e.g. floats in scientific notation. Instead of yaml.safe_load the default jsonargparse._loaders_dumpers.yaml_load could be used. Currently not public but could be made public.

[mauvilsa]

With #543 now it is possible to get the default yaml loader.