Constructed Inventory does not get passed extra vars - AWX

[jseifeddine]

Please confirm the following

• [X] I agree to follow this project's code of conduct.
• [X] I have checked the current issues for duplicates.
• [X] I understand that AWX is open source software provided for free and that I might not receive a timely response.
• [X] I am NOT reporting a (potential) security vulnerability. (These should be emailed to [email protected] instead.)

Bug Summary

No response

AWX version

Ansible AWX 23.5.2

Select the relevant components

• [ ] UI
• [ ] UI (tech preview)
• [ ] API
• [ ] Docs
• [ ] Collection
• [ ] CLI
• [X] Other

Installation method

docker development environment

Modifications

no

Ansible version

2.15.9

Operating system

Ubuntu

Web browser

No response

Steps to reproduce

Ansible AWX - constructed inventory doesn't get the extra_vars / survey variables passed to it...

Create "constructed inventory"

Set source vars for constructed inventory:

Run Job / Workflow Job (tested both) - with extra_vars or survey vars:

[WARNING]: * Failed to parse /runner/inventory/constructed.yml with auto plugin: failed to parse /runner/inventory/constructed.yml: Could not set play_customer for host HOST123: 'play_customer' is undefined. 'play_customer' is undefined . Could not set play_customer for host HOST234: 'play_customer' is undefined. 'play_customer' is undefined

On Ansible CLI (outside AWX) - it works perfectly.

ansible-playbook -i ./dynamic-mysql.yml -i ./constructed.yml playbooks/setup.yml -e "play_customer='Ferrari'" -l target

Expected results

play_customer variable being interpolated in constructed inventory

Actual results

[WARNING]: * Failed to parse /runner/inventory/constructed.yml with auto plugin: failed to parse /runner/inventory/constructed.yml: Could not set play_customer for host HOST123: 'play_customer' is undefined. 'play_customer' is undefined . Could not set play_customer for host HOST234: 'play_customer' is undefined. 'play_customer' is undefined

Additional information

I've done lot of testing and confirmed that the 'Control Plane' execution environment that runs the inventory source job for the constructed inventory, does not have the extra vars

All the other jobs in the workflow - do have the extra vars

[jseifeddine]

Please also see:

https://docs.ansible.com/ansible/latest/collections/ansible/builtin/constructed_inventory.html#parameter-use_extra_vars

[fosterseth]

@AlanCoding is AWX expected to support the use_extra_vars: True declaration?

[AlanCoding]

This is a valid point. The problem is that it runs ansible-inventory to save the content to the database before it is used in a job (ansible-playbook) at which time it is re-serialized from the database. There's not a 1-to-1 mapping of constructed inventories to jobs, so there's not a way to make this feature work without breaking other things. Maybe there's some envelope of parameters where it is set to update on launch and the plugin specifics this parameter, it would be okay to apply the job's extra_vars to the constructed inventory.

[jseifeddine]

Thanks @fosterseth and @AlanCoding for your response...

Is there any way for me to modify the cloned repo to add this in a quick fix ? Like the invocation of the constrcuted inventory sync job - anything at all ?

Would be fantastic, because at the moment, we're just pasting a massive comma seperate list into the Limit of the job ...

cheers, happy to spend time to investigate if you point me in the right direction :)

What would be ideal - and I'm not really sure why this isn't "a thing" - is using the extra vars -e while building a dynamic inventory... bypassing the constructed inventory

#   MySQL Dynamic Inventory
#   https://github.com/j-oss2023/mysql-dynamic-inventory # Custom Dynamic Plugin
#   
#   Required columns: 
#   'inventory_group', 'inventory_hostname'
#
#   TLDR:
#   Besides the required columns, each column will be assigned 
#   as an ansible hostvar dynamically.

plugin: mysql-dynamic-inventory
db_host: "{{secrets.db_host}}"
db_user: "{{secrets.db_user}}"
db_pass: "{{secrets.db_pass}}"
db_name: "{{secrets.db_name}}"
db_query: |
  SELECT 
    'motorvehicle_co' as inventory_group,
    host as inventory_hostname,
    address as ansible_host,
    account as customer from dealerships
  WHERE customer = "{{target_customer}}"

Then simply calling the playbook like so

ansible-playbook -i per_customer_mysql.yml setup.yml -e 'target_customer="Ferrari"'

That would rock! But I've read a lot of places that it's simply unattainable ...

Maybe i'm just coming at this at the wrong angle and it's giving me tunnel vision 😆 But I wish this was possible...

[jseifeddine]

Thanks @fosterseth and @AlanCoding for your response...

Is there any way for me to modify the cloned repo to add this in a quick fix ? Like the invocation of the constrcuted inventory sync job - anything at all ?

Would be fantastic, because at the moment, we're just pasting a massive comma seperate list into the Limit of the job ...

cheers, happy to spend time to investigate if you point me in the right direction :)

What would be ideal - and I'm not really sure why this isn't "a thing" - is using the extra vars -e while building a dynamic inventory... bypassing the constructed inventory

#   MySQL Dynamic Inventory
#   https://github.com/j-oss2023/mysql-dynamic-inventory # Custom Dynamic Plugin
#   
#   Required columns: 
#   'inventory_group', 'inventory_hostname'
#
#   TLDR:
#   Besides the required columns, each column will be assigned 
#   as an ansible hostvar dynamically.

plugin: mysql-dynamic-inventory
db_host: "{{secrets.db_host}}"
db_user: "{{secrets.db_user}}"
db_pass: "{{secrets.db_pass}}"
db_name: "{{secrets.db_name}}"
db_query: |
  SELECT 
    'motorvehicle_co' as inventory_group,
    host as inventory_hostname,
    address as ansible_host,
    account as customer from dealerships
  WHERE customer = "{{target_customer}}"

Then simply calling the playbook like so

ansible-playbook -i per_customer_mysql.yml setup.yml -e 'target_customer="Ferrari"'

That would rock! But I've read a lot of places that it's simply unattainable ...

Maybe i'm just coming at this at the wrong angle and it's giving me tunnel vision 😆 But I wish this was possible...

I've fixed the plugin, https://github.com/ansible/awx/issues/14868#issuecomment-1945208879

which now correctly templates vars in the inventory config file... so that is working now with ansible cli ...

but still - AWX bug fix / feature enhancment is needed to allow it to read those extra vars inside AWX...