awx
awx copied to clipboard
ansible
Allow Vault Credentials for Project SCM Type Inventories to process encrypted vault files
ISSUE TYPE
- Feature Idea
SUMMARY
Inventories sourced from project SCM do not currently support vault credentials. When using inventory plugins with secrets within the yaml spec files, there is no way to cleanly decrypt the vaulted file to allow the plugin to run its course.
Custom credentials are also not an option as AWX/Tower reserves all environment variables starting with ANSIBLE_, so there is no clear way to dynamically create a passwords file for vault from a custom credential.
As I just hit this blocker in my endeavour to get AWX up and running for our existing Ansible configuration: Any updates on this?
We ran into the same problems, but after reading this suggested solution: won't the encrypted values be stored as plain text as group and host variables?
I think you are right @coudenysj, but my hopes are that the devs find a solution to store the values encrypted after import, so you have to unlock them before usage. Just like SSH key passphrases. But that's just my humble non-dev point of view.
We are affected by this issue as well. Are there any plans for enhancements here?
awx is not fully providing the command option used in ansible-inventory. Storing host individual vault files is not possible, while it is working in command line.
It would be nice to have full command option support like :
ansible-inventory --help | grep ask-vault-pass
[--ask-vault-pass | --vault-password-file VAULT_PASSWORD_FILES]
--ask-vault-pass ask for vault password
This is a blocker for us too. It makes it very tricky for AWX to be used on an enterprise scale.
The workaround for this really sucks. I took my upstream branch and created a downstream dedicated to awx. This dedicated branch has the password to the Ansible vault file committed in plaintext in the repo and the ansible.cfg has vault_identity_list set.
This is really bad as now we've got extremely open credentials effectively in the clear. This feature is a must!
If this feature would be accepted, I wouldn't mind coding it myself if I could get some pointers on where to start.
We need one single issue to track this RFE (and it will be this), but it should be more specific. This is to request that vault credentials can be used in inventory imports to decrypt vars files.
That final noun "files" is important - meaning files encrypted by ansible-vault as described in the section "Encrypting files with Ansible Vault" in the Ansible docs https://docs.ansible.com/ansible/latest/user_guide/vault.html
The old issue was closed because one (preferred) method works, and we allow a new issue (this) for the not-working method. Here is a comment I left in that old issue describing the method that works https://github.com/ansible/awx/issues/223#issuecomment-416946662 and this corresponds to the "Encrypting individual variables with Ansible Vault" section in the Ansible docs link. This solution defers decryption of encrypted hostvars so that vault credentials associated with a job template can decrypt them.
Here is content I have in source control that can be used to follow those steps, with an SCM inventory import as part of the process:
https://github.com/AlanCoding/Ansible-inventory-file-examples/tree/master/vault/single_var_file
This works today, and it has worked for the last 4 years. This method avoids the thorny questions about security that come up with this issue:
We ran into the same problems, but after reading this suggested solution: won't the encrypted values be stored as plain text as group and host variables?
Agreed, it would be an unacceptable solution to just decrypt on import (although it may be better than plaintext in source control). Work needs to be put in to develop a feature design that could satisfy requirements.
To explain my title change - I added "to process encrypted vault files" so that this captures the missing noun I described in my last comment. If the vault credential were to decrypt individual hostvars, that would be a risky anti-feature. Implementation-wise, I don't even know if decrypting is the right solution, it might be that we create a new storage location where we put the content and write back out for playbook runs so this follows the same pattern.
@AlanCoding Would skipping encrypted files be an option during inventory processing? This would allow to use encrypted files like suggested in ansible best practices and not expose the actual data outside of a playbook run.
Yes, I expect that would be the solution. But the file content would have to be stored somewhere in the database, and we'd need to figure out exactly how to prepare them for the job run.
Curious, why would the file contents have to be stored in the database? After all the playbook run has access to the project files. FWIW I'd love to have a mode where AWX doesn't store variables at all (and just imports hosts & groups because I guess that is really needed). Would that be an even "easier" option? Can I somehow help with regards to the later goal?
I was able to get my inventory, that contains vault encrypted data, to sync from git. I had set the ANSIBLE_VAULT_PASSWORD_FILE: /runner/project/vault_pass in the "Details" tab/screen on the inventory source tab/screen. Then I had to exec on to container (i used the k8 dashboard) in the awx pod and find the directory where my project lives (a directory with a name that starts with '_' ) and echo "thepassword" > vault_pass. Terrible workaround I know.
Curious, why would the file contents have to be stored in the database? After all the playbook run has access to the project files.
This would be the case if you are using an SCM inventory that lives in the same directory as your playbook.
If your inventory comes from another project or another folder, then Ansible should parse the group_vars/ and host_vars/ adjacent to the inventory file as well as those folders adjacent to the playbook. If inventory-adjacent vars are not parsed, then expected variables will be lost. The related issue 2597 is a toggle to turn that off. Just want to clarify.
anu update on this?? I still can't import inventory due to secret/vault files within group_vars and host_vars dirs that pertain to the hosts file?
https://github.com/ansible/awx/issues/2597 is a viable near-term option, fully supporting those secrets (this issue) would be major work and is unlikely to happen.
I have recently ran into this old issue. I have found a workaround to decrypt the vault files during the inventory sync without adding the vault password to SVM or adding a static vault_file to the execution environment.
This workaround still has the problem of encrypted inventory variables being stored decrypted in AWX after sync.
Tested on AWX 9.1.1 and AWX 22.1.0
First, create a new credential type that will hold the encrypted vault password:
Input Configuration:
# One required string variable "vault_password" that will store a secret
fields:
- id: vault_password
type: string
label: Vault password
secret: true
required:
- vault_password
Injector Configuration:
# Create a file with the value of "file.template" and create an environment
# variable "vault_path" that will have as value the path to the created file.
file:
template: '{{ vault_password }}'
env:
vault_path: '{{ tower.filename }}'
After that create an instance of this new credential type with the vault password as vault_password.
Lastly add this new credential to the inventory source and adjust the ANSIBLE_VAULT_PASSWORD_FILE env variable in the inventory source pointing to the file containing the vault_password that will be created by the new credential.
# These are environment variables, which means these are
# referenced with shell syntax -> "$var" instead of "{{ var }}"
ANSIBLE_VAULT_PASSWORD_FILE: "$vault_path"
