Jinja expressions are not evaluated in (not only) credential variables

[mordekasg]

OS / Environment

Debian 12

Ansible version

ansible [core 2.14.17]
  config file = /home/test/host/ansible/ansible.cfg
  configured module search path = ['/home/test/host/ansible/local_modules']
  ansible python module location = /home/test/env.ansible/lib/python3.11/site-packages/ansible
  ansible collection location = /home/test/host/ansible/collections:/usr/share/ansible/collections
  executable location = /home/test/env.ansible/bin/ansible
  python version = 3.11.2 (main, May  2 2024, 11:59:08) [GCC 12.2.0] (/home/test/env.ansible/bin/python)
  jinja version = 3.1.4
  libyaml = True

Environment variables for strategy plugin

ANSIBLE_STRATEGY=mitogen_linear
ANSIBLE_STRATEGY_PLUGINS=/home/test/env.ansible/src/mitogen/ansible_mitogen/plugins/strategy

Mitogen version

v0.3.12

Problem

Jinja expressions are not evaluated by mitogen plugin

How to reproduce

Description

Ansible host: srv1 has credentials configured using Jinja expressions from vault/hostvars variables. Ansible host: srv2 has credentials configured directly as host variables.

Vault file

Path: vault.test.yml

ANSIBLE_LOCALHOST_SRVS_USER: "testusr"
ANSIBLE_LOCALHOST_SRVS_PASSWORD: "okmqwe123"
ANSIBLE_LOCALHOST_SRVS_BECOME_USER: "root"
ANSIBLE_LOCALHOST_SRVS_BECOME_PASSWORD: "okmqwe123"

Inventory file

Path: inventory/local/local.yml

all:
  children:
    test_srvs:
      hosts:
        srv1:
          host_ip: 127.0.0.1
          ansible_host: "{{ host_ip }}"
          ansible_user: "{{ ANSIBLE_LOCALHOST_SRVS_USER }}"
          ansible_password: "{{ ANSIBLE_LOCALHOST_SRVS_PASSWORD }}"
          ansible_become_user: "{{ ANSIBLE_LOCALHOST_SRVS_BECOME_USER }}"
          ansible_become_password: "{{ ANSIBLE_LOCALHOST_SRVS_BECOME_PASSWORD }}"
          ansible_ssh_user: "{{ ansible_user }}"
          ansible_ssh_password: "{{ ansible_password }}"
        srv2:
          ansible_host: 127.0.0.1
          ansible_user: testusr
          ansible_password: okmqwe123
          ansible_become_user: root
          ansible_become_password: okmqwe123
          ansible_ssh_user: testusr
          ansible_ssh_password: okmqwe123

Commands

regular user

ansible -e @vault.test.yml -i inventory/local/local.yml -m ping test_srvs

superuser - become

ansible -b -e @vault.test.yml -i inventory/local/local.yml -m ping test_srvs

Output

regular user

srv1 | UNREACHABLE! => {
    "changed": false,
    "msg": "EOF on stream; last 100 lines received:\nhostname contains invalid characters\r",
    "unreachable": true
}
srv2 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

superuser - become

srv1 | UNREACHABLE! => {
    "changed": false,
    "msg": "EOF on stream; last 100 lines received:\nhostname contains invalid characters\r",
    "unreachable": true
}
srv2 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Commands with verbose

regular user

ansible -e @vault.test.yml -i inventory/local/local.yml -m ping srv1 -vvvv

superuser - become

ansible -b -e @vault.test.yml -i inventory/local/local.yml -m ping srv1 -vvvv

Output

[mux 871642] 20:15:32.461360 D mitogen.parent: command line for Connection(None): ssh -o "LogLevel ERROR" -l "{{ ansible_user }}" -o "Compression yes" -o "ServerAliveInterval 30" -o "ServerAliveCountMax 10" -o "StrictHostKeyChecking no" -o "UserKnownHostsFile /dev/null" -o "GlobalKnownHostsFile /dev/null" -C -o ControlMaster=no -o ControlPersist=60s "{{ host_ip }}" /usr/bin/python3 [...]

Expected result

srv1 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
srv2 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Source of the problem

Ansible: https://github.com/ansible/ansible/commit/6d2d476113b3a26e46c9917e213f09494fbc0a13

Similar issues

Ansible

Paramiko plugin: https://github.com/ansible/ansible/issues/78509 Ansible: https://github.com/ansible/ansible/pull/76590

Mitogen

#905 #978 #1022 #1040 #1079 #1116