Unix shell script tactics - best practices style guide

This Unix shell script style guide helps us write better Unix shell script code for speed, security, stability, and portability.

This guide is by SixArm.com, a software consultancy. Our customers use shell scripts within a wide variety of Unix shells, so we aim for POSIX for simple scripts.

Highlights:

• Aim for POSIX when possible because of portability and standardization
• Protect scripts by using set flags such as set -euf
• Print output with printf not echo
• Trap signals and exit by using trap trap_exit EXIT
• Use a main() function
• Run executable with no file name extension
• Source with dot operator . not keyword source
• for arg do loop
• Parse options via while…case not getopts/getopt
• Version name: use semantic versioning
• Help: use a function and HERE document
• Date & time format: use UTC and ISO8601
• Booleans: use true and false
• Subshells: use parentheses $() not backticks
• Trace using set -x then set +x without printing
• Hunt for bugs by using ShellCheck
• Fix some bugs automatically, if you want, by using Shellharden

Recommendations (details of these are TODO):

• Quote liberally such as "$var" instead of just $var, for safety.
• Bulletproof scripts to handle characters such as a quote, newline, leading dash.
• Enable a user to customize commands by using env vars such as ${FOO:-foo}.
• Create temporary files by using mktemp instead of tempfile et. al.

Demo:

• Unix shell script tactics demo

Topics

Argument parsing:

• Arg parse
• Parse options via while and case not getopts or getopt

Environment variables:

• Test if set or unset
• $FUNCNAME function name

Directories:

• Script directory: use realpath or cd
• Log directory: use $LOG_HOME
• Data directory: use $XDG_DATA_HOME
• Cache directory: use $XDG_CACHE_HOME
• Configuration directory: use $XDG_CONFIG_HOME
• Runtime directory: use $XDG_RUNTIME_HOME
• Temporary directory:: use mktemp

Files:

• Temporary file using mktemp and trap
• Find files with filter for permission denied
• Find files with special characters
• Find files with readable permissions
• Find files with executable, perm, test, exec

References:

• Writing safe shell scripts
• Rich’s sh (POSIX shell) tricks
• Google Shell Script Style Guide
• Filenames and Pathnames in Shell: How to do it Correctly
• Standard Command-Line Options
• How to do things safely in bash
• pure sh bible & pure bash bible

Conventions:

• Program name using a string or basename
• Git top level directory using rev-parse
• Functions: out, err, die, big, log, now, sec, zid, cmd, etc.
• Assert functions: assert_empty, assert_equal, assert_match
• sec() function portability

Control flow statements:

• While loop with index counter
• Do while loop
• Case statement that skips option dash flags

Functions:

• Export function
• Number functions
• Array functions
• URL encode
• URL decode

Command snippets:

• awk match_between

Compatibilities:

• readlink on macOS behaves differently
• realpath is` not available on older macOS default

PostgreSQL psql:

• psql helpers