update_fact will prevent variable replacement under certain conditions

[ipc-zpg]

SUMMARY

under certain conditions, update_facts will prevent variables being substituted within variables it manipulates

ISSUE TYPE

• Bug Report

COMPONENT NAME

ansible.utils.update_fact

ANSIBLE VERSION

ansible [core 2.11.4]
  config file = /Users/pookey/.ansible.cfg
  configured module search path = ['/Users/pookey/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/Cellar/ansible/4.5.0/libexec/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/pookey/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.9.7 (default, Sep  3 2021, 04:31:11) [Clang 12.0.5 (clang-1205.0.22.9)]
  jinja version = 3.0.1
  libyaml = True

COLLECTION VERSION

ansible-galaxy collection list ansible.utils

# /opt/homebrew/Cellar/ansible/4.5.0/libexec/lib/python3.9/site-packages/ansible_collections
Collection    Version
------------- -------
ansible.utils 2.4.0

# /Users/pookey/.ansible/collections/ansible_collections
Collection    Version
------------- -------
ansible.utils 2.4.3

CONFIGURATION

OS / ENVIRONMENT

Tested on OSX and Linux

STEPS TO REPRODUCE

---

- hosts: localhost
  vars:
    env_name: moo
    datadog_checks:
      mysql:
        - host: "{{ env_name }}.moo"
      codedeploy:
        logs: []
  tasks:
    - debug:
        var: datadog_checks.mysql[0].host
    - name: update fact
      ansible.utils.update_fact:
        updates:
          - path: "datadog_checks['codedeploy']['logs']"
            value: "{{ datadog_checks['codedeploy']['logs'] + ['/path/to/logfile'] }}"
      register: new_dd
    - name: replace the datadog_checks array
      set_fact:
        datadog_checks: "{{ new_dd.datadog_checks }}"
    - debug:
        var: datadog_checks.mysql[0].host

EXPECTED RESULTS

the second debug should print 'moo.moo', in the same way the first debug does.

ACTUAL RESULTS

I get {{ env_name }}.moo as the output.

❯ ansible-playbook ./test.yaml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] *****************************************************************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************************************************************
ok: [localhost]

TASK [debug] *********************************************************************************************************************************************************************
ok: [localhost] => {
    "datadog_checks.mysql[0].host": "moo.moo"
}

TASK [update fact] ***************************************************************************************************************************************************************
changed: [localhost]

TASK [replace the datadog_checks array] ******************************************************************************************************************************************
ok: [localhost]

TASK [debug] *********************************************************************************************************************************************************************
ok: [localhost] => {
    "datadog_checks.mysql[0].host": "{{ env_name }}.moo"
}

PLAY RECAP ***********************************************************************************************************************************************************************
localhost                  : ok=5    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[cidrblock]

@ipc-zpg Thanks for this, it took a few minutes to remember the subtleties of vars vs. facts in this case.

This is not entirely unexpected, at the time the update_fact task is run the datadog_checks variable has not had its references resolved yet. This can be seen in the following playbook:

- hosts: localhost
  gather_facts: false
  vars:
    env_name: moo
    datadog_checks:
      mysql:
        - host: "{{ env_name }}.moo"
      codedeploy:
        logs: []
  tasks:
    - name: Show delayed resolution of variables
      set_fact:
        env_name: A cow says
  
    - name: What does a cow say?
      debug:
        var: datadog_checks

TASK [What does a cow say?] ******************************************************************************************************************
ok: [localhost] => {
    "datadog_checks": {
        "codedeploy": {
            "logs": []
        },
        "mysql": [
            {
                "host": "A cow says.moo"
            }
        ]
    }
}

The workaround for this is to resolve all the references in datadog_checks by setting a fact referring to it, and updating that:

---
- hosts: localhost
  gather_facts: false
  vars:
    env_name: moo
    datadog_checks:
      mysql:
        - host: "{{ env_name }}.moo"
      codedeploy:
        logs: []
  tasks:

    - name: Set a fact, this will resolve references
      set_fact:
        resolved_datadog: "{{ datadog_checks }}"

    - name: Update the fact
      ansible.utils.update_fact:
        updates:
          - path: "resolved_datadog['codedeploy']['logs']"
            value: "{{ resolved_datadog['codedeploy']['logs'] + ['/path/to/logfile'] }}"
      register: updated_datadog
    
    - name: Show the new fact
      ansible.builtin.debug:
        var: updated_datadog

    - name: Replace the datadog_checks array
      set_fact:
        datadog_checks: "{{ updated_datadog.resolved_datadog }}"
    - debug:
        var: datadog_checks.mysql[0].host

TASK [debug] *********************************************************************************************************************************
ok: [localhost] => {
    "datadog_checks.mysql[0].host": "moo.moo"
}

In your example, ansible passes the unresolved var to the task, so that is what is being returned.

Let me know if this helps

-Brad

[cidrblock]

@ipc-zpg Just checking in to see if ^^^ helped explain the behaviour

[alice-rc]

So I have another example. It seems that if the path includes a variable representing a number, it also will not work.

This works fine:

- ansible.utils.update_fact:
    updates:
      - path: "vm_info['value']['cdroms']['3000']['backing']['type']"
        value: CLIENT_DEVICE

This works fine:

- ansible.utils.update_fact:
    updates:
      - path: "vm_info['value']['cdroms']['3000']['backing'][{{ lastpart }}]"
        value: CLIENT_DEVICE
  vars:
    lastpart: type

This fails:

- ansible.utils.update_fact:
    updates:
      - path: "vm_info['value']['cdroms'][{{ cdrom_id }}]['backing']['type']"
        value: CLIENT_DEVICE
  vars:
    cdrom_id: '3000'

TASK [ansible.utils.update_fact] ***********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 3000
fatal: [XXX06-XX1-XXX29]: FAILED! => {"changed": false, "msg": "Error: the key '3000' was not found in {'3000': {'start_connected': False, 'backing': {'auto_detect': True, 'device_access_type': 'EMULATION', 'type': 'HOST_DEVICE'}, 'allow_guest_control': True, 'ide': {'master': True, 'primary': True}, 'label': 'CD/DVD drive 1', 'state': 'NOT_CONNECTED', 'type': 'IDE'}}."}

[Rockawear]

I'm hitting this bug, any workaround?

[danielleshoemake]

I also have the issue of not being able to use a variable that is set to equal an index number, but in my case, the error seems to indicate that update_fact interprets all variables as strings.

This works

- ansible.utils.update_fact:
    updates:
      - path: installation_media["{{sqlserver_version}}"][0]['product_id']
        value: ''

These both fail with the same error

- ansible.utils.update_fact:
    updates:
      - path: installation_media["{{sqlserver_version}}"]["{{idx}}"]['product_id']
        value: ''
  vars:
    idx: 0

- ansible.utils.update_fact:
    updates:
      - path: installation_media["{{sqlserver_version}}"]["{{idx | int}}"]['product_id']
        value: ''
  vars:
    idx: 0

The error

The error was: TypeError: list indices must be integers or slices, not str
FAILED! => {"changed": false, "msg": "Error: the key '0' was not found in....