Escape/normalise special chars in paths: spaces, &, etc.

[u-fischer]

Running l3build check in latex2e/base in a miktex installation which has a space in the installation path fails with

Running l3build with target "check" with configuration build
Script file C:/Users/Nililand2020/AppData/Local/Programs/MiKTeX not found
Running l3build with target "check" with configuration config-1run
Script file C:/Users/Nililand2020/AppData/Local/Programs/MiKTeX not found

[josephwright]

Could we cover this in normalize_path() or abspath()? Might also deal with some of the other issues with file names - @zauguin any thoughts?

[josephwright]

I've broadened out the title: if we handle special chars, we should do it in one place then extend that as/when needed (and make sure it's applied when required).

[zauguin]

Could we cover this in normalize_path() or abspath()?

Not in abspath since we often need absolute paths even if we don't end up passing it on the command line. normalize_path might work but escapepath seems like a more fitting candidate. But we have to completely change the use of that function. Currently this gets applied for all the ...dir variables which will probably cause more issues than it solves since it leads to paths which are not usable from Lua. Instead it should be applied just before all system calls.

[zauguin]

But there's also the problem that path escaping in general is not a well defined problem on Windows. Different programs sometimes need paths to be escaped in different ways, there are some paths which can't be escaped for some programs and for many programs the escape rules aren't documented... But IIRC most of these things only show up once paths containing quotes get involved...

[josephwright]

Sounds like we may need an additional function to apply normalisation in the 'right' places (perhaps within the various file copy, remove, etc., functions?).

On normalisation, I think we add such entries as we know we require to get those basic (Command Prompt) functions to work.

[josephwright]

@zauguin I think we are done here?

[josephwright]

I think we are broadly done here.