trans slow on WSL and Cygwin due to bash's read slowness - Solution

[qwer1304]

I confirmed that performance hit in WSL and Cygwin is due to bash poor performance in read statements $ time read -r -d '' TRANS_PROGRAM < build/trans.awk

real 0m2.473s user 0m0.109s sys 0m2.313s

There's an alternative implementation of heredoc (from https://stackoverflow.com/questions/1167746/how-to-assign-a-heredoc-value-to-a-variable-in-bash scroll to 2nd answer) that uses cat instead and that is much faster.

VAR=$(cat <<'END_HEREDOC' abc'asdf" $(dont-execute-this) foo"bar"'' END_HEREDOC )

heredoc with cat $ time build/trans -b -t ru hello Здравствуйте

real 0m1.691s user 0m0.938s sys 0m0.469s

heredoc with read $ time build/trans -b -t ru hello Здравствуйте

real 0m4.268s user 0m1.031s sys 0m2.828s

There's a problem though with a \ at the end of a line: bash command substitution treats this as line continuation eventhough 'EOF' is used, so you end up with RSins( instead of RS\ ins( Shell behaves correctly. PS zsh doesn't have such an issue

The solution is to: 0.turn on lastpipe option This is required since we assign a variable inside a pipe which is run in a subshell and thus is unavailable once we're out of the subshell and back to the main shell There're alternatives here http://tldp.org/LDP/abs/html/gotchas.html but I didn't explore them further 1.sed the heredocs to change trailing \ to \\ 2.pipe the result into a variable through command substitution using cat (not read!) 3.sed the variable to change trailing \\ back to \ and pipe into gawk

The attached build.awk implements all this (remove txt suffix) build.awk.txt

Here're the timings: $ time build/trans -b -t ru hello Здравствуйте

real 0m1.426s user 0m0.813s sys 0m0.469s

Hope you find this useful

[qwer1304]

Small improvement: lastpipe option works when job control is not active. To ensure that, add set +m before shopt -s lastpipe

[soimort]

Now I can observe a great performance superiority of cat over read. Thanks for the info.

Unfortunately this requires some nontrivial tweaks of the build script. I'll put this change onto the TODO list.