JSON Merges Style Inherits by Default

[Averylamp]

Inherits Spec

Want to move from our custom inherits scheme to a scheme such as JSON merge to allow for more flexibility when merging different parameter types. Hoping to use a operator schema based merge, but inline in our yamls. Here's the spec:

---

Inherits Keyword Options

The special yaml field that triggers inherits has the following form

# Generic form with Optional `<Operator>` and `<Scope>` delimited with `$` and `|` respectively
inherits$<Operator>|<Scope>:

# simple inherits with default options (Operator:overwrite)
inherits:

# Operator specified
inherits$concat:

# Scope specified
inherits|root:

# Operator and Scope specified
inherits$replace|match:

---

Inherits field file options

There are two formats of how the inherits field works. The inherits: field can take in either a list of yamls to inherit from, or a singular yaml file to inherit. If there are a list of files, the latter files take precedence over earlier files (if overwriting), but essentially each file should be applied with the inherits operator sequentially.

# List of inherits, applied sequentially
inherits:
  - inherit1.yaml
  - inherit2.yaml

# singular value inherits only values underneath the field
inherits: inherit_field.yaml

By default, relative filepaths to the file that calls inherits are used. If a absolute path is give, starting with /, absolute filepaths are used.

Each inherited file can recursively inherit other files (using a relative or absolute path).

Do not worry about circular inherits (maybe throw an error if possible).

---

Inherits Scope

When inheriting from other yamls, there are two options that decide where in the other yaml to start inheriting from.

• match (default)
• root All future examples are done with the updates operator (default operator)

---

Match example

1.yaml

produce:
  tomatoes:
    inherits: 2.yaml
  potatoes: almost ripe

2.yaml

produce:
  tomatoes: ripe

out.yaml

produce:
  tomatoes: ripe
  potatoes: almost ripe

---

Root

Example 1

1.yaml

produce:
  tomatoes:
    inherits: 2.yaml
  potatoes: almost ripe

2.yaml

tomatoes: ripe

out.yaml

produce:
  tomatoes: 
    tomatoes: ripe
  potatoes: almost ripe

Note, in this example, the keys are duplicated, because the root yaml is loaded as the value of tomatoes:

Example 2

1.yaml

produce:
  tomatoes:
    inherits: 2.yaml
  potatoes: almost ripe

2.yaml

ripe

out.yaml

produce:
  tomatoes: ripe
  potatoes: almost ripe

This produces the expected, as 2.yaml is loaded directly from the root and update is applied to the tomatoes value

---

The inherits field can be placed anywhere within a dictionary, and will only inherit/operate within its current scope from that point in the dictionary to the leaves. For example:

1.yaml

tomatoes:
  inherits|match: 2.yaml
potatoes: almost ripe

2.yaml

tomatoes: ripe
potatoes: planted

out.yaml

tomatoes: ripe
potatoes: almost ripe

---

Operators

There are a couple of operators we want to support

• update (default)
• concat (much like update, but concats list fields)
• replace (replaces the value directly without updates, removes non existing values)

A lot of what differentiates operators is how they deal with lists or leaf values.

---

Update Operator

The update operator is the default operator. It will update a dictionary of values

1.yaml

produce:
  inherits$update|root:
    - 2.yaml
  tomatoes: 
    number: 12
    type: cherry
    status: ripe
    tags:
     - organic
     - fertilized
  potatoes: 
    type: russell

2.yaml

tomatoes: 
  number: 13
  tags:
    - gmo
potatoes:
  status: dying

out.yaml

produce:
  tomatoes: 
    number: 13
    type: cherry
    status: ripe
    tags:
     - gmo
  potatoes: 
    type: russell
    status: dying

---

Concat Operator

The concat operator differs from the update operator in leaf list values. Note that instead of replacing the entire list, the lists are concatenated together. Also note, this only applies to list items, but not other + implementing items such as int or str

1.yaml

produce:
  inherits$update|root:
    - 2.yaml
  tomatoes: 
    number: 12
    type: cherry
    status: ripe
    tags:
     - organic
     - fertilized
  potatoes: 
    type: russell

2.yaml

tomatoes: 
  number: 2
  tags:
    - gmo
potatoes:
  status: dying

out.yaml

produce:
  tomatoes: 
    number: 2
    type: cherry
    status: ripe
    tags:
     - organic
     - fertilized
     - gmo
  potatoes: 
    type: russell
    status: dying

---

Replace Operator

The replace operator replaces the entire value with the inherited values.

Note: it does not make sense to use inherits$replace with a list of yamls as only the last will be the value

Example 1

1.yaml

produce:
  inherits$replace|root:
    - 2.yaml
  tomatoes: 
    number: 12
    type: cherry
    status: ripe
    tags:
     - organic
     - fertilized
  potatoes: 
    type: russell

2.yaml

tomatoes: 
  number: 2
  tags:
    - gmo

out.yaml

produce:
  tomatoes: 
    number: 2
    tags:
      - gmo

Example 2

1.yaml

produce:
  inherits$replace|root:
    - 2.yaml
    - 3.yaml
  tomatoes: 
    number: 12
    type: cherry
    status: ripe
    tags:
     - organic
     - fertilized
  potatoes: 
    type: russell

2.yaml

tomatoes: 
  number: 2
  tags:
    - gmo

3.yaml

None

out.yaml

produce: None

---

Other Gotchas

---

Example 1 (No match found)

1.yaml

produce:
  inherits:
    - 2.yaml
  tomatoes: 
    number: 12
    type: cherry
  potatoes: 
    type: russell

2.yaml

tomatoes: 
  number: 2
  tags:
    - gmo

out.yaml

produce:
  tomatoes: 
    number: 12
    type: cherry
  potatoes: 
    type: russell

Nothing is updated as the produce: key is not matched.