Object of type HostVarsVars is not JSON serializable

[pinkbubba]

I'm getting error

[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin
(<ansible.plugins.callback.awx_display.CallbackModule object at
0x7f54a8551c70>): Object of type HostVarsVars is not JSON serializable

when running the below playbook in AWX (ansible-runner)

- name: test 
  hosts: localhost
  tasks:
    - debug: 
        msg: 
         - "{{ item.inventory_hostname}}"
      loop: "{{ ansible_play_hosts
        | default([])
        | map('extract', hostvars) 
        | list 
        }}"

ansible-runner --version 2.1.4.dev2

python --version Python 3.8.13

[pinkbubba]

Callback Exception: 
  File "/usr/local/lib/python3.8/site-packages/ansible/executor/task_queue_manager.py", line 450, in send_callback
    method(*new_args, **kwargs)
   File "/usr/local/lib/python3.8/site-packages/ansible_runner/display_callback/module.py", line 495, in v2_runner_item_on_ok
    with self.capture_event_data('runner_item_on_ok', **event_data):
   File "/usr/lib64/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
   File "/usr/local/lib/python3.8/site-packages/ansible_runner/display_callback/module.py", line 100, in capture_event_data
    event_context.dump_begin(sys.stdout)
   File "/usr/local/lib/python3.8/site-packages/ansible_runner/display_callback/events.py", line 197, in dump_begin
    self.cache.set(":1:ev-{}".format(begin_dict['uuid']), begin_dict)
   File "/usr/local/lib/python3.8/site-packages/ansible_runner/display_callback/events.py", line 72, in set
    partial_data = json.dumps(value, cls=AnsibleJSONEncoderLocal)
   File "/usr/lib64/python3.8/json/__init__.py", line 234, in dumps
    return cls(
   File "/usr/lib64/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
   File "/usr/lib64/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
   File "/usr/local/lib/python3.8/site-packages/ansible_runner/display_callback/events.py", line 52, in default
    return super(AnsibleJSONEncoderLocal, self).default(o)
   File "/usr/lib64/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__}

[eqrx]

@shanemcd @AlanCoding any thoughts on that?

[AlanCoding]

I ran that playbook, and I can't get it to give any error. Could you provide more details to reproduce the issue?

With the thing that it's looping over, it's likely sensitive to the contents of your inventory, or maybe the adjacent host_vars/ folder (vars plugin). I don't doubt that you might have hit this with some given inventory content, but I would be guessing to try to figure out how.

[sivel]

It looks like the JSON encoder class is missing support for Mapping which core has:

https://github.com/ansible/ansible-runner/blob/d467662f585f5bb6ed2c5e8d57f130772760031a/ansible_runner/display_callback/callback/awx_display.py#L76

I didn't specifically test this playbook, but I'd guess it has something to do with the item from the loop that is getting serialized somehow.

[AlanCoding]

Well, we can't import from ansible in that context, and Mapping is an internal import. That is, then, conditionally imported depending on python version, and for what runner supports that resolves to collections.abc.Mapping. We could do a somewhat superficial unit test for this and trust that it fixes the issue, ideally, or we could get a playbook reproducer.

[sivel]

Well, we can't import from ansible in that context

Yep, I wasn't recommending that. Just that you could import collections.abc.Mapping and mirror the logic that core has, by casting Mapping to dict.

[Akasurde]

resolved_by_pr #1181

[AlanCoding]

I tried again, and still can't reproduce this. Could someone hold my hand for getting the above error?

added file test/fixtures/projects/debug/project/hostvars_as_mapping.yml

- name: Test using hostvars as mapping
  connection: local
  hosts: localhost
  gather_facts: false
  tasks:
    - debug:
        msg:
         - "{{ item.inventory_hostname}}"
      loop: "{{ ansible_play_hosts
        | default([])
        | map('extract', hostvars)
        | list
        }}"

run

ansible-runner run test/fixtures/projects/debug/ -p hostvars_as_mapping.yml