hyperfine
hyperfine copied to clipboard
sharkdp
Bug ~ quoting? failure on windows-platform
In example, for bat:
C:\>bat.exe --paging=always --pager="cmd.exe /c more.com" README.md
│ File: README.md
1 │ uutils coreutils
2 │ ================
3 │
4 │ [](https://discord.
│ gg/wQVJbvJ)
5 │ [](https://github.com/uutils/coreutils/blob/master/LICENSE)
6 │ [](https://app.fos
│ sa.io/projects/git%2Bgithub.com%2Fuutils%2Fcoreutils?ref=badge_shield)
7 │ [](https://github.com/Aaronepower/tokei)
8 │ [](https://deps.rs/repo/github/uutils/coreuti
│ ls)
9 │
10 │ [](https://travis-ci.org/uutils/coreutils)
11 │ [](https://ci.appveyor.com
│ /project/Arcterus/coreutils)
12 │ [](https://cirrus-ci.com/github/uutils/cor
│ eutils/master)
When testing with hyperfine:
C:\>hyperfine "bat.exe --paging=always --pager=\"cmd.exe /c more.com\" README.md"
Benchmark #1: bat.exe --paging=always --pager="cmd.exe /c more.com" README.md 0
Error: Command terminated with non-zero exit code. Use the '-i'/'--ignore-failure' option if you want to ignore this. Alternatively, use the '--show-output' option to debug what went wrong.
but, using a command-string without internal quoting (by passing arguments to bat via the environment):
env BAT_PAGER="cmd.exe /c more.com" hyperfine "bat.exe --paging=always README.md"
Benchmark #1: bat.exe --paging=always README.md 0
Time (mean ± σ): 199.9 ms ± 51.9 ms [User: 10.3 ms, System: 6.8 ms] 0
Range (min … max): 179.8 ms … 347.5 ms 10 runs
Warning: The first benchmarking run for this command was significantly slower than the rest (347.5 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.
The text of the target command shown by hyperfine looks correct, but the target is not being executed as displayed.
This is likely a shell quoting issue.
Thank you for reporting this.
So what are the actual quoting/escaping rules for cmd.exe? I seem to remember that they are highly non-intuitive (for a Unix user). Can
"bat.exe --paging=always --pager=\"cmd.exe /c more.com\" README.md"
be passed to other programs without any problem?
You're correct; parsing command lines on Windows is complicated and quirky.
And the rules changed in 2008.
David Deley has a good explanation of the various rules and changes (with examples) "How Command Line Parameters Are Parsed".
If you're handing the command line off to cmd, you should be able to just mechanically escape every character with ^ (eg, cmd.exe /d/c ^e^c^h^o ^"^t^h^i^s^ ^>^ ^w^o^r^k^s^").
If you're handing off directly to another windows process, I think it's ok to assume the 2008+ parsing behavior, except that rust is currently using the older, pre-2008, parsing behavior (which is extra "quirky"). But it does look like that is likely to change. I'm sure there must be mechanical translation recipe that would work for the direct process approach, but I haven't been able to create or find it.
Command is struggling with this as well.
That's why I fell back to cmd.exe /d/c ... for env in coreutils.
- ref: How Command Line Parameters Are Parsed @@ http://archive.is/9jM4q
- ref: Discussing env::args on Windows (GH:rustlang/rust#44650)
- ref: Discussion about Powershell command line escaping
- ref: MSDN ~ Parsing C command line args
- ref: Python's work-around
- ref:
Commanddoes not escape arguments as expected on windows
Thank you for the detailed explanation and for all the references, especially "How Command Line Parameters Are Parsed". I didn't know it was that complicated :frowning:
I'm inclined to accept the cmd.exe /d/c ^…^… workaround for hyperfine, if you think that this would solve most issues.
Alternatively, would it help to use PowerShell instead of cmd.exe? This could be easily tested via hyperfines --shell option (if PowerShell also uses the syntax /c <cmd>).
Alternatively, would it help to use PowerShell instead of
cmd.exe? This could be easily tested viahyperfines--shelloption (if PowerShell also uses the syntax/c <cmd>).
I don't think PowerShell is a solution at this point. The initial design was not well thought out and it's also struggling with command line escaping and still evolving.
cmd.exe is, in comparison, a known, relatively stable, problem space.
It looks like the resolution of https://github.com/rust-lang/rust/issues/29494 (a new raw_arg method on Windows) could help us resolve this issue. It's currently only available on nightly though.
It looks like the resolution of rust-lang/rust#29494 (a new
raw_argmethod on Windows) could help us resolve this issue. It's currently only available on nightly though.
👍🏻 I've been keeping an eye on that issue as well.
Looks like this might be stabilized in 1.61, so it will take a few more months (at least).
In the meantime, you might be able to use the new -N/--shell=none option of hyperfine to disable the shell completely.
FYI @rivy
raw_arg has been stabilized for a while now: https://doc.rust-lang.org/std/os/windows/process/trait.CommandExt.html#tymethod.raw_arg
Someone with access to a Windows machine might be able to resolve this, ideally backed up by some new (regression) tests.
Hello, I can take a shot a this. I've being meaning to get more into OSS and rust, and I run mostly on windows, specially at work.
Pretty cool tool, btw. It's my go-to for performance tests 🥇
