Support ash as its own dialect

[mmlb]

For bugs

• Rule Id (if any, e.g. SC1000):
• My shellcheck version (shellcheck --version or "online"):
• [x] I tried on shellcheck.net and verified that this is still a problem on the latest commit
• [ ] It's not reproducible on shellcheck.net, but I think that's because it's an OS, configuration or encoding issue

For new checks and feature suggestions

• [x] shellcheck.net (i.e. the latest commit) currently gives no useful warnings about this
• [x] I searched through https://github.com/koalaman/shellcheck/issues and didn't find anything related

I commented on https://github.com/koalaman/shellcheck/issues/209#issuecomment-285814514 that I've hit a couple of things that ash supports that dash/sh does not, it would be nice to have shellcheck recognize this.

Here's a snippet or screenshot that shows the problem:

#!/bin/sh

[[ -e /etc/hosts ]] && echo hosts exists

Here's what shellcheck currently says:

Line 3:
[[ -e /etc/hosts ]] && echo hosts exists
^-- SC2039: In POSIX sh, [[ ]] is undefined.

Here's what I wanted or expected to see:

Example showing ash is ok with this

[manny@manny ~]$ docker run --rm -ti ubuntu:xenial sh -c '[[ -e /etc/hosts ]] && echo hosts exists'
sh: 1: [[: not found
[manny@manny ~]$ docker run --rm -ti alpine:3.5 sh -c '[[ -e /etc/hosts ]] && echo hosts exists'
hosts exists
[manny@manny ~]$ docker run --rm -ti alpine:3.2 sh -c '[[ -e /etc/hosts ]] && echo hosts exists'
hosts exists

[nihilus]

Actually your error is due to the shebang, use /bin/dash or tell shellcheck by cli which interpreter you are using; 'shellcheck -s dash' in your case.

[mmlb]

@nihilus I don't see how telling shellcheck to use dash helps here. dash does not support the feature I'm looking to use in ash.

[manny@manny ~]$ shellcheck - <<EOF
#! /bin/sh
[[ -e /etc/hosts ]] && echo hosts exists
EOF

In - line 2:
[[ -e /etc/hosts ]] && echo hosts exists
^-- SC2039: In POSIX sh, [[ ]] is undefined.

[manny@manny ~]$ shellcheck -s dash - <<EOF
#! /bin/sh
[[ -e /etc/hosts ]] && echo hosts exists
EOF


In - line 2:
[[ -e /etc/hosts ]] && echo hosts exists
^-- SC2169: In dash, [[ ]] is not supported.

[manny@manny ~]$ shellcheck -s bash - <<EOF
#! /bin/sh
[[ -e /etc/hosts ]] && echo hosts exists
EOF

I added bash since ash is exhibiting same behavior as bash here.

[mmlb]

Hmmm but know I'm learning that it seems like there are multiple implementations of ash ? At least https://www.in-ulm.de/~mascheck/various/ash is a reference of a few. I'm using busybox's ash as part of alpine linux as can be seen in the OP. Maybe the way to handle would be finer grained bashims? So we could do something like:

#! /bin/sh
# shellcheck shell=bash
# shellcheck disable=SCB0042 <- process substitution

or maybe an enable directive along with not quite so fine grained bashims:

#! /bin/sh
# shellcheck enable=SCB0001 # pipefail
# shellcheck enable=SCB0002 # substring and replacement parameter expansion
# shellcheck enable=SCB0003 # bash builtins
...

both seem like they would be a lot of work though

[nihilus]

'shellcheck -s dash -e SC2169' is what I use for bug squashing with busybox ash. @mmlb as noted shellcheck doesn't recognise the shell and defaults to POSIX sh, use the options above.

[mmlb]

@nihilus yup I've been doing something similar for most of my scripts, just adding the disable=SC2169 where needed but today I came upon the problem with this approach.

#!/bin/ash
#shellcheck shell=dash

foo=abcd
[[ "$foo" == *bc* ]] && echo yay

gives me SC2169. But because shellcheck does not understand the ash dialect properly and we disable SC2169 then we lose out on detecting that ash is going to try to match the literal string '*bc*' instead of what the user probably thought would happen (glob pattern match, a bashism it seems).

[fredcadete]

I have the same issue (more like a wish :)) as @mmlb . Examples of things that are supported in ash but shellcheck complains because they are not in dash:

• ==
• [[ ]]
• &>, >&
• Using source instead of .
• ulimit -c, ulimit -n
• echo -e, echo -n
• read -s, read -n
• string indexing, a la ${var:0:1}
• substring replacement, a la ${var/pat/replacement}

Furthermore, some features could depend on compile options for busybox. From the source code it looks like arithmetic loops could be supported or not.

[mmlb]

I think the biggest impediment to supporting ash as its own dialect is the apparently different versions of ash that exist https://en.wikipedia.org/wiki/Almquist_shell. Maybe support

#!/*/ash
#shellcheck shell=busybox|bsd's|...

?

[Strykar]

+1 for ash support of the Busybox version. This would be great for troubleshooting scripts on OpenWRT.

[Herst]

@Strykar The challenge with Busybox ash is that it can be compiled with wildly varying configs, but maybe certain assumptions can be made, e.g. that ASH_BASH_COMPAT to support certain bashisms is always enabled.

[Strykar]

@Herst I see what you mean - https://github.com/openwrt/openwrt/blob/d5a38725f84f52618a7c4cb17e8bfac40a17854a/package/utils/busybox/Config-defaults.in https://github.com/openwrt/openwrt/blob/696c511fb480be04a0ab4563b06b5e1b0d7eb684/package/utils/busybox/config/shell/Config.in

Then there's more options for file, networking and other utils - https://github.com/openwrt/openwrt/search?q=busybox&unscoped_q=busybox

[mmlb]

There's also alpine linux to take into account, https://github.com/alpinelinux/aports/blob/397f0cd902eaadce3b74e1e3021e941d9bb6ba72/main/busybox/busyboxconfig#L1101. So far the 2 main users of busybox ash seem to enable BASH compat.

[ElvenSpellmaker]

I'd like to add my twopenn'orth here, ash for alpine support would be great, another one to add to this list is that source is supported in ash, but not dash.

EDIT: I suppose that comes under enabling BASH compat. But not all of bash is supported in ash obviously, such as heredoc strings (<<<) or process substitution, < <(foo).

[ashthespy]

Has someone already made a list of directives to disable for ash with ASH_BASH_COMPAT for busybox scripts?

[Olf0]

Has someone already made a list of directives to disable for ash with ASH_BASH_COMPAT for busybox scripts?

Maybe the list of ash internal options set, when ASH_BASH_COMPAT is set, is helpful: see lines 193 - 231 of ash.c

[tspearconquest]

Hello, I'd like to ask for a documentation update.

The below code:

#!/bin/busybox sh
set -o pipefail

Results in:

In script.sh line 3:
set -o pipefail
       ^------^ SC3040: In POSIX sh, set option pipefail is undefined.

However, busybox added support for pipefail in 2010 -- so it would be useful (imho) for the docs to indicate something about using # shellcheck shell=bash as an alternative to setting it to dash when using a busybox binary that you know has ASH_BASH_COMPAT enabled. Surely in cases where shellcheck is used in pre-commit git hooks, this would be great to find easily, because busybox is used quite a lot in Docker images based on Alpine.

[Olf0]

There's also alpine linux to take into account, …

There is also SailfishOS to take into account, https://github.com/sailfishos/busybox/blob/master/rpm/busybox-sailfish.config, which also sets CONFIG_ASH_BASH_COMPAT=y.

But all this is not useful, because …

• the Busxybox ash has a ton of other compile time configuration options, such as POSIX and SUSv3 compatibility (which is IMO much more relevant than BASH compat).
• at first sight I could not find a proper way to determine which of these many configuration options have been set, hence what shellcheck might test for.
• the Busybox ash is just one of the many (≥ 13!) diverging ash variants (of which ≥ 6 are still maintained), each with its own set of configuration options. For details, see https://www.in-ulm.de/~mascheck/various/ash/

The gist of that is:

1. Generic tests for ash must only test for the common ash baseline, i.e., features which all variants of ash support, regardless of the chosen compile time options. AFAICS, that is almost an empty set.
2. Anything beyond these generic ash tests requires a scheme to determine at runtime, which ash variant is running the shellcheck tests.
3. Testing features of ash, which may or may not be enabled during compile time require a scheme to determine, which features of an ash variant were enabled.

Anything else will return a lot of false complaints by shellcheck for many ash variants and configurations. So all proponents of creating ash-specific tests IMO should try to find ways to determine a specific ash variant and its compile time options at runtime.

[tspearconquest]

Hi! I don't disagree with anything you said, and I hope that can clarify that my ask was just simply to list out in the documentation that there are other options than dash.

This page specifically mentions the shebang !#/bin/ash and debian.

However as you can see evidenced by other users on this thread, SC2187 is caused by #!/bin/busybox sh when a # shellcheck shell= is not specified as well.

My ask is to update the SC2187 page be more generic in describing the check. It's not specific to just #!/bin/ash even though busybox is a variant of ash, so the documentation should be generic enough to list out alternative shebangs which may trigger a complaint from this check and alternative # shellcheck shell= directives users may try.

[mcassaniti]

Could I suggest a different path? Instead of having a new shell=busybox-sh_bash-compat or similar, why not provide a script that tests what features work for the target environment and have the script generate something like below:

#shellcheck shell=bash
#shellcheck disable=...

Is this idea feasible? Maybe it's shell=sh with disable=... but you get the idea.

[port19x]

Could I suggest a different path? Instead of having a new shell=busybox-sh_bash-compat or similar, why not provide a script that tests what features work for the target environment and have the script generate something like below:

#shellcheck shell=bash
#shellcheck disable=...

Is this idea feasible? Maybe it's shell=sh with disable=... but you get the idea.

Not really feasible to implement in an automated fashion

[phil-hands]

When considering automated detection of the busybox variant in use, keep in mind that one might be writing scripts on a system that may not even have busybox installed, targeting some other system (e.g. writing scripts for debian-installer, or OpenWRT, on a full desktop).

I wonder if the busybox folk could be persuaded generate a shellcheck enable/disable snippet as part of their build process, that would reflect whatever settings were enabled at build time. That could then be included as part of the documentation when busybox is distributed in binary form, and there would be no need to try to discover the features available.

[koalaman]

Busybox Ash support was added by @grische in #2868. Thanks!

[SuperSandro2000]

@koalaman can you do a new release so that the change propagates into distros? Thanks

[grische]

@SuperSandro2000 #2918