Enable the use of anchor and aliases for the checks yml

[jrsalazar14]

Currently if we add a new alias to the check yml file, the scan and tests run but we are seeing a Error on the CLI ERROR | Invalid section header "variables"

Would be nice to be able to use anchor and aliases without facing errors on the CLI.

[jmarien]

SODA-1200

[m1n0]

hello @jrsalazar14 , could you please post an example of what are you trying to do that causes errors and also the use cases you are trying to solve with this? thanks!

[jrsalazar14]

@m1n0 Sure!

var1: &var1 ['1', '2', '3']
checks for DATABASE:
  - invalid_count(COLUMN1) = 0:
      name: Valid Values Check
      valid values: *var1
  - invalid_count(COLUMN2) = 0:
      name: Valid Values Check
      valid values: *var1
  - invalid_count(COLUMN3) = 0:
      name: Valid Values Check
      valid values: *var1

By using aliases and anchor we can use the same value on different checks, here I am passing the var1 alias as valid values to different checks without having to rewrite them.

[m1n0]

@jrsalazar14 thanks for the example! Indeed, soda-core does not support yaml variables at the moment to make parsing of the sodaCL DSL a bit easier and possible to have strict grammar.

If you have an idea about how yaml vars would work well with the sodaCL grammar a contribution is always welcome of course, feel free to open a PR if you have a solution in mind!

Alternatively, you could pre-process the checks file, use variables/templating, even jinja if you want, and then run the scan programatically