Add an option to treat `/bin/sh` interpreter as `busybox`

[srcshelton]

Now that busybox is a recognised shell dialect, it would be useful to be able to optionally configure (via command-line options and/or .shellcheckrc?) #!/bin/sh shebang lines encountered in analysed scripts to set the language variant to busybox by default instead of POSIX.

Without this option, it seems that the only way to analyse scripts for use with busybox sh would be to add an annotation to every single script?

[brother]

Documentation seems to be lacking but it works already.

shellcheck --shell=ash scriptfile.sh

[srcshelton]

I was thinking more generally:

Say I have a directory with a mixture of scripts, some with interpreters of /bin/sh (which I need to be treated as ash/busybox) and some of /usr/bin/env bash (or equivalent) which I still want to analyse bash scripts unchanged - I don't want to force everything to ash, but I do want to analyse the sh scripts as ash/busybox rather than posix.

e.g. what I feel's needed is a global remapping option to say "whenever you would have performed the analysis as shell a, instead use shell b".

Perhaps a --remap-sh=<posix|ash|busybox> or --remap posix=<ash|busybox> or similar?

[brother]

Ok understood. Not really anything to do with the recent merge then. Maybe retitle it a bit. Seems like an interesting idea.

[srcshelton]

The relevance now is that a new Docker Hub container-image was pushed and the Change Log shows new added support for busybox sh (… although how this differs from the pre-existing support for ash, I don't know!)

[grische]

I also think that your feature request is independent of the busybox sh feature. /bin/sh being interpreted as POSIX shell sounds still the way to go, not dash, ash or others.

If you have write access to those files, simply doing something like a

sed -i 's,#!/bin/sh,#!/bin/busybox sh,' file1 file2 ...

would probably fix it properly.

although how this differs from the pre-existing support for ash, I don't know!

Default busboy sh supports considerably more features than ash. Hence I recommend to pick one or the other, not mixing them. See #2865 for examples.

[srcshelton]

Default busboy sh supports considerably more features than ash. Hence I recommend to pick one or the other, not mixing them. See #2865 for examples.

I had no idea they weren't the same!

I also think that your feature request is independent of the busybox sh feature. /bin/sh being interpreted as POSIX shell sounds still the way to go, not dash, ash or others.

If you have write access to those files, simply doing something like a

sed -i 's,#!/bin/sh,#!/bin/busybox sh,' file1 file2 ...

would probably fix it properly.

This still requires changing the source script, though, and making it incompatible for systems which don't have /bin/busybox (rather than a shellcheck update which would allow /bin/sh to actually be busybox, POSIX, bash, or anything else compatible via a command-line option!)

[grische]

I'm not sure I understand your use case here.

You have scripts that are written for POSIX (i.e. have a /bin/sh shebang) and you want to interpret them as something else? Why?

POSIX feature set is the smallest subset of all shell features of the before mentioned busybox sh, ash and bash. Following this train of thought for the other ones: bash supports all features of busybox sh which in turn supports all features of ash. But the opposite is not true. (This won't work for other shells like ksh for example).

So if a script passes shellsheck as POSIX, it will also pass as any other interpreter. On the other hand, if a script passes as bash, it might pass neither as ash or busybox sh (but it could of course).

A shebang should always write at least the shell with the smallest sets of features required to run it (or one with more), but never with less features to avoid running into crashes or undefined behaviour.

[srcshelton]

…because, depending on the distribution, /bin/sh might be POSIX, it might be busybox, it might be ash or dash (or something else again).

And it might be enough to say "add a directive to the script in order to tell shellcheck which variant to evaluate against" - but it might also be a huge timesaver to be able to say "For a directory of unmodified scripts without specific shellcheck directives, if you see /bin/sh then treat it as", for example, "busybox rather than POSIX by default".

(On Ubuntu, for example, it might be useful to set the /bin/sh syntax to validate against as dash?)

Because even though POSIX is indeed the base standard that any other valid script will also be compatible with, it's also possible to have many scripts which have interpreter /bin/sh but which use (say) busybox extensions, and should be validated without warnings even if these aren't POSIX compliant.

What I'm suggesting is that if I have a directory of 100 scripts with /bin/sh as an interpreter which are compliant with busybox rather than POSIX, I should be able to check these with shellcheck --assume-sh=busybox … unchanged rather than having to first modify every one of these 100 hypothetical files to add an interpreter directive.

I take your final point about using the appropriate interpreter to the standard used, but the vast majority of pre-existing scripts won't adhere to this - and I find myself fixing-up third-party scripts as often as writing new ones.

[grische]

Let me see if I understand this correctly:

1. You have scripts that are not strictly POSIX compliant and have a /bin/sh shebang. Those scripts "work" because the local /bin/sh is a symlink to /bin/bash for example
2. Those scripts run exclusively on that platform, i.e. will always "work" (as long as the distro doesn't change the symlink).
3. You can't fix the shebang in those scripts

Hence you want to test against bash instead of POSIX to find, other errors beyond the wrong shebang?

[srcshelton]

Without getting too engrossed in the semantics (/bin/sh doesn't have to be a symlink, for example, and I don't think there are hard guarantees in respect to exclusivity - plus I'm not saying scripts should be validated against bash specifically, simply that having a configurable choice would be helpful!), that's broadly the situation yes.

The situation I've seen - and I'm surprised if others haven't - is that people will write scripts (without running shellcheck on them 😩), and they'll apparently work, so these will be published/distributed. But it turns out that their /bin/sh wasn't actually POSIX and so these scripts contain syntax or extensions which work correctly on, say, Ubuntu - but fail elsewhere where /bin/sh isn't dash (c.f. https://wiki.ubuntu.com/DashAsBinSh), as an example.

Potentially any script with an interpreter of /bin/sh which is analysed as or includes anything other than POSIX should generate a warning to alert developers to this situation?

[brother]

It kinda sounds like you try to get a feature into a project to avoid doing the work at hand. Converting everything to strict POSIX or some such obnoxious task.

shellcheck is meant to be a guide and something to hold hands with when it gets slippery under the feet. Using a directive to ignore false positives is a nice and good thing. And so on.

Designing stuff with busybox as target and then have sh as or something else as shebang and not wanting to deal with these things is a tad bit strange. You can specify the shell to validate against already, I do understand that will not cut it for your case at this moment but maybe the good thing would be to rewrite the validation in your end and execute shellcheck with the correct shellname on the command line, that would mostly be the same as you suggest. And when the mammoth task of rewriting the sources to behave in a more classic way you can remove the hacks. Or maybe someone has closed this task with a PR fulfilling your hopes.

[srcshelton]

Not making any moral judgement against anyone's situation or motivation, I'll point out that across different distributions, the binary or symlink providing /bin/sh might actually conform to busybox sh, or dash… or perhaps even original POSIX sh.

And I'm not even suggesting that shellcheck should change its assumption that an interpreter of /bin/sh does always mean POSIX.

I'm merely suggesting that a mechanism to override this assumption on a per-host basis would be a helpful (and easily implemented?) feature.

(… there might even be mileage in having shellcheck interrogate /bin/sh and automagically choose the appropriate variant - although this wouldn't want to be the default, as it risks getting different results from different distros in, e.g., CI situations. But --sh-variant=auto could be made a manual option to give the most potentially appropriate output for the situation each environment shellcheck finds itself invoked in)

[koalaman]

I definitely see the problem, though it seems overly specialized to add a feature that assumes that /bin/sh is a specific shell when the point of /bin/sh is to refer to a generic shell. However, I think you could get much the same effect and ergonomics through one of

• Adding shell=busybox in a .shellcheckrc in the relevant directory (and giving any non-busybox scripts an annotation to override it)
• Specifying #!/bin/busybox sh as the interpreter to be explicit