Add yaml serializer for addict dictionary

[jkrzemin]

Using kwargs to update root on kadet input right now can lead to quite a frustrating and a bit misleading error

yaml.representer.RepresenterError: ('cannot represent an object', {'name': 'http-api', 'port': 8080, 'targetPort': 8080, 'protocol': 'TCP'})

It's because a dictionary is of Dict type which is unknown to yaml serializer, which misleadingly (in context of this error message) has the same text representation as regular python dictionary.

[ramaro]

@jkrzemin thanks for reporting. Can you paste a snippet that is causing this? Just to make sure I understand it correctly

[jkrzemin]

Ok, so here is a class definition

(venv) ➜  kapitan_issue_example cacat inventory/classes/my_component.yml
parameters:
  kapitan:
    compile:
      - output_path: kadet_output
        input_type: kadet
        output_type: yaml
        input_paths:
          - components/my_component

Here is python source for component

(venv) ➜  kapitan_issue_example cacat components/my_component/__init__.py
from kapitan.inputs import kadet

inv = kadet.inventory()

class SomeResource(kadet.BaseObj):
    def body(self) -> None:
        self.root.update({
            'key_for_dictionary_parameter': {
                'inner_key': self.kwargs.dict_param
                }
        })


def main():
    output = kadet.BaseObj()
    output.root['my_component'] = SomeResource(dict_param=inv.parameters.dict_parameter)
    return output

Which causes

(venv) ➜  kapitan_issue_example   kapitan compile
Unknown (Non-Kapitan) Error occurred
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/Users/jkrzemin/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/kapitan/targets.py", line 465, in compile_target
    input_compiler.compile_obj(comp_obj, ext_vars, **kwargs)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/kapitan/inputs/base.py", line 54, in compile_obj
    self.compile_input_path(input_path, comp_obj, ext_vars, **kwargs)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/kapitan/inputs/base.py", line 69, in compile_input_path
    self.compile_file(
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/kapitan/inputs/kadet.py", line 156, in compile_file
    fp.write_yaml(item_value)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/kapitan/inputs/base.py", line 125, in write_yaml
    yaml.dump(obj, stream=self.fp, indent=indent, Dumper=PrettyDumper, default_flow_style=False)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/__init__.py", line 290, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/__init__.py", line 278, in dump_all
    dumper.represent(data)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 58, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/yaml/representer.py", line 231, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', {'foo': {'bar': 'car'}})
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/kapitan/targets.py", line 136, in compile_targets
    [p.get() for p in pool.imap_unordered(worker, target_objs) if p]
  File "/private/tmp/kapitan_issue_example/venv/lib/python3.8/site-packages/kapitan/targets.py", line 136, in <listcomp>
    [p.get() for p in pool.imap_unordered(worker, target_objs) if p]
  File "/Users/jkrzemin/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 865, in next
    raise value
yaml.representer.RepresenterError: ('cannot represent an object', {'foo': {'bar': 'car'}})


('cannot represent an object', {'foo': {'bar': 'car'}})

An interesting thing I've found during preparing this snippet is that if I'd assign self.kwargs.dict_param to key_for_dictionary_parameter instead of key_for_dictionary_parameter['inner_key'] it works properly. I didn't go through the source, but it smells a bit like a shallow check during compile step or YAML serialization.

Edit: I've forgotten about target spec, here it is.

(venv) ➜  kapitan_issue_example cacat inventory/targets/my_target.yml
classes:
  - common
  - my_component

parameters:
  target_name: my_target
  dict_parameter:
    foo:
      bar: car

[jkrzemin]

@ramaro does this snipped provide information you need?

[ramaro]

@jkrzemin thanks, this makes it clearer! It is indeed a case of assigning self.kwargs.dict_param to key_for_dictionary_parameter, that's by design. However I do agree that requiring assigning from self.kwargs might not always be obvious or desired, so maybe we can relax that somehow.