Ignoring file that is a broken symbolic link produces "[Errno 2] No such file or directory"

[thomasdstewart]

Hi,

When running yamllint on a directory of files if a file is a broken symbolic link it produces "[Errno 2] No such file or directory" even if the file is set to be ignored. I was able to reproduce using latest yamllint[0] along with a small test environment[1] and by running "yamllint ." to show the error[2].

Kind Regards Tom

[0]

$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip install yamllint
$ yamllint --version
yamllint 1.26.2
$

[1]

$ mkdir test; cd test
$ ln -s not_there ignore_file.yaml
$ ls -l ignore_file.yaml 
lrwxrwxrwx 1 thomas thomas 9 Aug  5 10:45 ignore_file.yaml -> not_there
$ cat .yamllint 
---
extends: default

ignore: |
  ignore_file.yaml
$

[2]

$ yamllint .
[Errno 2] No such file or directory: './ignore_file.yaml'
$

[adrienverge]

Hello @thomasdstewart, thanks for the report.

I can confirm the problem. Any clean contribution to fix this (and include non-regression tests) would be welcome!

Out of curiosity: do you have a use-case where you have broken .yaml symlinks, and want to keep them?

[thomasdstewart]

Hi @adrienverge,

Indeed filing an issue is a bit strong wording for what is more like a feature request from some random on the Internet and by far not a demand for you to fix. :)

Apologies, yes some context would have been useful. I've got a public git repo with a set of Ansible playbooks, roles, templates, vars, vaults, etc. The vault files are in the standard locations but are symlinks to the real files in a private repo, eg separating the public and private data. eg:

$ file group_vars/all/vault.yml group_vars/all/vault.yml: symbolic link to ../../safe/group_vars/all/vault.yml $ file safe/group_vars/all/vault.yml safe/group_vars/all/vault.yml: ASCII text $

To side step this, in my CI I just swapped out "yamllint ." with: find . -type f ( -name ".yml" -o -name ".yaml" ) ! -name 'vault.yml' | xargs yamllint

Kind Regards Tom

[thomasdstewart]

What I didn't really funny understand is that that it is an exit(-1) exception at cli.py:207. It's not skipping over the file, it's completely stopping and so further files are not scanned. I was thinking it probably best for is_file_ignored() to be called before io.open() is called, as that is where the exception occurs rather than currently that is called as part of linter.run()? Further more FileNotFoundError should probably produce a warning about it as well? eg

diff --git a/yamllint/cli.py b/yamllint/cli.py
index cb87350..788cd3b 100644
--- a/yamllint/cli.py
+++ b/yamllint/cli.py
@@ -204,6 +204,8 @@ def run(argv=None):
         try:
             with io.open(file, newline='') as f:
                 problems = linter.run(f, conf, filepath)
+        except FileNotFoundError as e:
+            print(e, file=sys.stderr)
         except EnvironmentError as e:
             print(e, file=sys.stderr)
             sys.exit(-1)

Produces:

$ python3 -m yamllint yamllint/t
yamllint/t/vault.yaml
  2:2       warning  missing starting space in comment  (comments)

[Errno 2] No such file or directory: 'yamllint/t/nonexistant.yml'
yamllint/t/valid.yaml
  3:2       warning  missing starting space in comment  (comments)

Rather than:

$ yamllint t
t/vault.yaml
  2:2       warning  missing starting space in comment  (comments)

[Errno 2] No such file or directory: 't/nonexistant.yml'

[noonedeadpunk]

Out of curiosity: do you have a use-case where you have broken .yaml symlinks, and want to keep them?

My usecase are git submodules as symlink is aimed to be pointed to the content inside submodule but I don't really want to test what's in that another included repo, since it has it's own testing.

So basically implementing option not to follow symlinks at all would be great.

[tom0010]

Out of curiosity: do you have a use-case where you have broken .yaml symlinks, and want to keep them?

My usecase are git submodules as symlink is aimed to be pointed to the content inside submodule but I don't really want to test what's in that another included repo, since it has it's own testing.

So basically implementing option not to follow symlinks at all would be great.

We have exactly this too, I ended up pulling down the submodules for now, but would be great to see this implemented.

[TomyLobo]

I have another use case: To indicate that a file is being modified by another instance, Midnight Commander's editor creates symlinks named ".#<filename>", targetted at "<user>@<FQDN>.<PID>". I'd like to exclude those as well.

[james-powis]

For what it is worth, its a simple fix... Submitted a PR https://github.com/adrienverge/yamllint/pull/646

[benformosa]

I'm also having this issue. My repo has a symlink to a file in a submodule, which isn't present when I run yamllint in CI.

I've added an ignore rule for the symlink, and use this command: yamllint --list-files . | xargs yamllint

[james-powis]

I think it is worth arguing that yamllint traversing symlinks, in general, should be configurable... In one use case the same few yaml files are symlinked into thousands of directories, due to being named the same as the source file, i cannot exclude, and the result is yamllint runs against each and every symlink, effectively linting the same file thousands of times.

Where the broken symlink becomes problematic is when a specific directory is no longer needed, it is preserved by moving it into a different directory with a different relative depth but still within scope of the root of the target when running yamllint. As a result, we hit this bug. I have monkeypatched yamllint with the changes in PR #646 however i also had to extend it further to skip over symllinks.

I have created https://github.com/adrienverge/yamllint/pull/647 to demonstrate how complex it can be to handle symlinks well, in a dry manner. Ultimately my though is to entirely exclude symlinks and if so desired individually target the target file instead of the symlink.

[adrienverge]

I've added an ignore rule for the symlink, and use this command: yamllint --list-files . | xargs yamllint

Nice trick :+1: (while waiting for a proper fix)

Where the broken symlink becomes problematic is when a specific directory is no longer needed, it is preserved by moving it into a different directory with a different relative depth but still within scope of the root of the target when running yamllint. As a result, we hit this bug.

I agree this should be fixed. Thanks for your pull request.

[MrMegaNova]

Thanks for the PR, I hope #646 gonna be merged ASAP