Automatically ignore ansible vault files

[tamere-allo-peter]

Ansible vaults are YAML files which are not parsable because they are encrypted with a password.

An example Ansible vault look like this :

$ANSIBLE_VAULT;1.1;AES256
1643039736532396535663733313030306436333431313465653962333739613331
...

When launching yamllint on this file we've got :

jerome@OPT17844:~$ yamllint --version
yamllint 1.26.3
jerome@OPT17844:~$ yamllint this-is-an-ansible-vault.yml
this-is-an-ansible-vault.yml
  1:1       warning  missing document start "---"  (document-start)

jerome@OPT17844:~$

Unfortunately adding the document start marker to such files would render the vault file unreadable by the ansible-vault command :

jerome@OPT17844:~$ cat this-is-a-modified-ansible-vault.yml
---
$ANSIBLE_VAULT;1.1;AES256
1643039736532396535663733313030306436333431313465653962333739613331
jerome@OPT17844:~$ 

jerome@OPT17844:~$ ansible-vault view this-is-a-modified-ansible-vault.yml 
Vault password: 
ERROR! input is not vault encrypted data for this-is-a-modified-ansible-vault.yml
jerome@OPT17844:~$ 

So the attached commit makes yamllint ignore Ansible vault files :

jerome@OPT17844:~$ yamllint this-is-an-ansible-vault.yml
jerome@OPT17844:~$ 

As an alternative, such files could be decrypted on the fly before being linted, but this would be much more complex.

[coveralls]

Coverage increased (+0.2%) to 99.066% when pulling a7db5585e2d401de799fd022ccba51b8e0227f97 on tamere-allo-peter:ignore-ansible-vaults into 40cab7f999b7d8f4f143f22f6932126274238e0d on adrienverge:master.

[adrienverge]

Hello Jérôme, thanks for this proposal.

Yamllint is dedicated to general-purpose YAML, it is not meant to adapt to specificities of Ansible, Symfony, GitHub Actions, Cloud-init, or any other third-party project. For this reason, this change is not acceptable.

As an Ansible user myself, I can suggest either:

• Don't use .yaml or .yml extension for your vaults. These are not YAML file, rather AES-encrypted files.
• Use ignore in your yamllint config to automatically exclude those files.

[tamere-allo-peter]

Too bad, because IMHO this prevents one from mass batch linting .yml files which might be foreign to the user wanting to lint them.

So maybe another option, more generic, would be for yamllint to automatically skip files that can't be YAML, like vault files, or jpeg images for examples (where there's a traceback currently), or whatever file format that can't be YAML.

[DimitriPapadopoulos]

But then how do you make the difference between a broken YAML file and a file that is not a YAML file?

This is a linter for YAML files. If you run the linter on files that are not YAML files (even if they use .yaml or .yml extensions) I do expect an error.

[andrewimeson]

maybe @tamere-allo-peter's suggestion (automatically skip files that seemingly can't be YAML) should be a configurable options to ignore unparsable files rather than error on them. The problem with that is that in the Ansible vault example, the file is treated as a valid YAML file unless we add special detection logic like the PR does, and that means we still have to commit to maintaining that.

I'm not sure that every time there is a disagreement about changing behavior we should just reimagine it as a non-default option, but I do think it might make sense here.

[adrienverge]

But then how do you make the difference between a broken YAML file and a file that is not a YAML file?

@DimitriPapadopoulos I agree!

@andrewimeson your proposal for a new option makes sense, but if the user needs to opt-in for this new option, it might be even simpler to use the already-existing configuration ignore:

ignore: |
  my-vaults/*.yaml

[andrewimeson]

@adrienverge the problem with the ignore approach is that they need to know based on filename what to ignore. In all of my use cases, that has been easy. I suspect that there are use cases driving this request where users can't predict the names of the files.

[andrewimeson]

The Ansible Vault example parses as valid YAML with a single empty key(?). That does make more general approaches to skipping-on-error, or skipping on >0 bytes (and not ---/...) and no YAML data harder.

python3 -c \
  'import sys, yaml; yaml.dump(yaml.safe_load(sys.stdin), sys.stdout)' << 'EOF'
$ANSIBLE_VAULT;1.1;AES256
1643039736532396535663733313030306436333431313465653962333739613331
EOF