ci: bump shfmt to 3.5.1, simplify CI setup

[kolyshkin]

1. Bump shfmt to v3.5.1. Release notes: https://github.com/mvdan/sh/releases

2. Since shfmt v3.5.0, specifying -l bash (or -l bats) is no longer necessary. Therefore, we can use shfmt to find all the files. Add .editorconfig to ignore vendor subdirectory.

3. Use shfmt docker image, so that we don't have to install anything explicitly. This greatly simplifies the shfmt CI job. Add localshfmt target so developers can still use a local shfmt binary when necessary.

[kolyshkin]

Rebased; bumped shfmt to 3.4.3.

@AkihiroSuda @thaJeztah PTAL

[kolyshkin]

CI is failing because of Azure Ubuntu mirror issues

E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/libc/libcap2/libcap-dev_2.32-1_amd64.deb Connection failed [IP: 52.252.75.106 80]

will restart it later.

[kolyshkin]

So, as I pointed out in https://github.com/opencontainers/runc/pull/3379#discussion_r833610510, we need shfmt 3.5.0 (not yet released) to further simplify things and use docker, so let's set this to pause for now.

[kolyshkin]

OK, shfmt 3.5.0 is out as of 8 days ago, but the docker image is still not :(

[kolyshkin]

OK, shfmt 3.5.0 is out as of 8 days ago, but the docker image is still not :(

And the reason is https://github.com/mvdan/sh/issues/860

I'm going back to fetching the binary.

[kolyshkin]

This only took a few months 🤣 @opencontainers/runc-maintainers PTAL (easy to review)

[thaJeztah]

oh! CI is failing? Let me check

[thaJeztah]

mvdan/shfmt:v3.5.1
-w cannot be used on standard input
make: *** [Makefile:168: shfmt] Error 1
Error: Process completed with exit code 2.

[kolyshkin]

mvdan/shfmt:v3.5.1
-w cannot be used on standard input
make: *** [Makefile:168: shfmt] Error 1
Error: Process completed with exit code 2.

Weird, I checked it locally (with podman) and it works.

Ah! We are using local shfmt to generate the list of files, and the one inside the docker image to work on them.

Hmm, I am not quite sure how to fix this to use the in-the-image shfmt for both.

[kolyshkin]

Hmm, since this is a single binary image, we need to run two containers -- first to get the file list, second to use it. I'm a bit confused.

[kolyshkin]

Or don't use shfmt -f, replacing it with something else.

[kolyshkin]

OK I looked into the logic of shfmt -l and it is somewhat complicated:

• it walks the directory tree;
• prints all files which have \.(sh|bash|mksh|bats|zsh)$ suffix;
• for other files found, drop those that contain a dot . in their names;
• for other files, check if their first line matches the ^#!\s?/(usr/)?bin/(env\s+)?(sh|bash|mksh|bats|zsh)(\s|$) regex.

So, I am not going to recreate this functionality here.

Perhaps we can just go back to downloading a binary, wdyt @thaJeztah?

[kolyshkin]

OK, git ls-files | grep this | grep that is good enough for now. Unlike shfmt -f, it does not look for shebangs, but the one file we have without the shell suffix is contrib/completions/bash/runc, and we can explicitly add it to the regexp.

I'd still love to use shfmt -f but what I have now will suffice.

[kolyshkin]

OK, git ls-files | grep this | grep that is good enough for now. Unlike shfmt -f, it does not look for shebangs, but the one file we have without the shell suffix is contrib/completions/bash/runc, and we can explicitly add it to the regexp.

I'd still love to use shfmt -f but what I have now will suffice.

Solved this one via .editorconfig.

PTAL @thaJeztah

[kolyshkin]

Rebased, resolved merge conflicts, addressed a nit from @AkihiroSuda

@thaJeztah PTAL

[AkihiroSuda]

Needs rebase

[kolyshkin]

Rebased; PTAL @thaJeztah @AkihiroSuda

[kolyshkin]

CI failure is a flake, see https://github.com/opencontainers/runc/issues/3540; CI restarted.