Windows can print color escape codes in a shell that doesn't support colors

[ArneBachmannDLR]

What's up?

It's impossible to interpret errors in a standard cmd shell, but the CompileOptions do not offer a color:bool=False option.

    SyntaxError: [31mError:[0m
       [38;5;246m╭[0m[38;5;246m─[0m[38;5;246m[[0m:4:16[38;5;246m][0m
       [38;5;246m│[0m
     [38;5;246m4 │[0m [38;5;249m [0m[38;5;249m [0m[38;5;249m [0m[38;5;249m [0m[38;5;249m [0m[38;5;249m [0m[38;5;249m [0m[38;5;249m [0m[38;5;249mf[0m[38;5;249mi[0m[38;5;249ml[0m[38;5;249mt[0m[38;5;249me[0m[38;5;249mr[0m[38;5;249m [0masset_id[38;5;249m [0m[38;5;249m=[0m[38;5;249m=[0m[38;5;249m [0m[38;5;249m$[0m[38;5;249mr[0m
     [38;5;240m  │[0m                ────┬───
     [38;5;240m  │[0m                    ╰───── Unknown name
    [38;5;246m───╯[0m

[max-sixty]

Thanks for the report. It shouldn't be coloring when not writing to an compatible terminal, but I can believe that it's making a mistake.

What's the full repro? Here's what I see:

 python -c "import pyprql; pyprql.compile('asdf')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
SyntaxError: Error:
   ╭─[:1:1]
   │
 1 │ asdf
   │ ──┬─
   │   ╰─── Unknown name
───╯

[ArneBachmannDLR]

In a mambaforge environment in a cmd.exe on Windows, no extras enabled:

C:\Users\bach_ar>activate 311

(311) C:\Users\bach_ar>python -c "import pyprql; pyprql.compile('asdf')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
SyntaxError: ←[31mError:←[0m
   ←[38;5;246m╭←[0m←[38;5;246m─←[0m←[38;5;246m[←[0m:1:1←[38;5;246m]←[0m
   ←[38;5;246m│←[0m
 ←[38;5;246m1 │←[0m asdf
 ←[38;5;240m  │←[0m ──┬─
 ←[38;5;240m  │←[0m   ╰─── Unknown name
←[38;5;246m───╯←[0m

But maybe there's some environment variables or packages installed that default to using colored output?

For now I'm using the Windows Terminal app instead, which supports colored output even for cmd.exe sessions :-)

[max-sixty]

Thanks. One debugging step — can we try `python -c "import pyprql; pyprql.compile('asdf') > error.log 2>&1" and see whether that produces escaped output?

[ArneBachmannDLR]

Besides the fact that I'm not using pyprql:

python -c "import prql_python; prql_python.compile('asdf')" > error.log 2>&1

(311) C:\Users\bach_ar>type error.log
Traceback (most recent call last):
  File "<string>", line 1, in <module>
SyntaxError: Error:
   \u256d\u2500[:1:1]
   \u2502
 1 \u2502 asdf
   \u2502 \u2500\u2500\u252c\u2500
   \u2502   \u2570\u2500\u2500\u2500 Unknown name
\u2500\u2500\u2500\u256f

[max-sixty]

Thanks! This does seem like a bug. I'll see if we can get any guidance upstream.

I don't have a windows box to test on, but if @eitsupi has any spare cycles and wants to try and repro, that would be great!

[max-sixty]

Hi @ArneBachmannDLR — would you be able to run these two commands (from https://github.com/rust-cli/anstyle/issues/111)

$ cargo install anstyle-query --examples --root anstyle-query
$ anstyle-query/bin/report
clicolor: None
clicolor_force: false
no_color: false
term_supports_ansi_color: true
term_supports_color: true
truecolor: true
enable_ansi_colors: None
is_ci: false

[ArneBachmannDLR]

Well not that easy...

C:\Users\bach_ar>cargo install anstyle-query --examples --root anstyle-query
    Updating crates.io index
  Downloaded anstyle-query v1.0.0
  Downloaded 1 crate (8.6 KB) in 0.59s
  Installing anstyle-query v1.0.0
    Updating crates.io index
  Downloaded windows-targets v0.48.1
  Downloaded windows_x86_64_msvc v0.48.0
  Downloaded windows-sys v0.48.0
  Downloaded 3 crates (3.3 MB) in 2.11s (largest was `windows-sys` at 2.6 MB)
   Compiling windows_x86_64_msvc v0.48.0
error: linker `link.exe` not found
  |
  = note: program not found

note: the msvc targets depend on the msvc linker but `link.exe` was not found

note: please ensure that Visual Studio 2017 or later, or Build Tools for Visual Studio were installed with the Visual C++ option.

note: VS Code is a different product, and is not sufficient.

error: could not compile `windows_x86_64_msvc` (build script) due to previous error
error: failed to compile `anstyle-query v1.0.0`, intermediate artifacts can be found at `C:\Users\bach_ar\AppData\Local\Temp\cargo-installIq385d`

(update the MS Build Tools)

C:\Users\bach_ar>set PATH=C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\Hostx86\x64;%PATH%

C:\Users\bach_ar>cargo install anstyle-query --examples --root anstyle-query
    Updating crates.io index
  Installing anstyle-query v1.0.0
    Updating crates.io index
   Compiling windows_x86_64_msvc v0.48.0
error: linking with `link.exe` failed: exit code: 1181
  |
  = note: "link.exe" "/NOLOGO" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\rustcm3q8Hs\\symbols.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.0.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.1.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.10.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.11.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.12.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.13.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.14.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.15.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.2.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.3.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.4.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.5.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.6.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.7.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.8.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.build_script_build.848c8543e7b8ed02-cgu.9.rcgu.o" "C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.28vqtltngo9us1pl.rcgu.o" "/LIBPATH:C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\deps" "/LIBPATH:C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd-f54c5a8dc7e7febf.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libpanic_unwind-92f7800729779464.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_demangle-6da5127b1b379d8e.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd_detect-efa74c8d8702db5b.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libhashbrown-e9ed0b4fbfe1d158.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_alloc-23c6870c75efea7d.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libminiz_oxide-19857ab63c6bf99c.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libadler-0d8b8ed2a2fb22e9.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunwind-3a3d97b2ce691588.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcfg_if-60ba5e33f92355d7.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liblibc-94c0a2e24d7696be.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liballoc-edd8efb6a2cb3252.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_core-cb8dd18d30da7a99.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcore-75c0b134deed8bd9.rlib" "C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-96d2ffc8ffc7b9b5.rlib" "kernel32.lib" "advapi32.lib" "bcrypt.lib" "kernel32.lib" "ntdll.lib" "userenv.lib" "ws2_32.lib" "kernel32.lib" "msvcrt.lib" "/NXCOMPAT" "/LIBPATH:C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "/OUT:C:\\Users\\bach_ar\\AppData\\Local\\Temp\\cargo-installIomCyX\\release\\build\\windows_x86_64_msvc-d6e71885dca539a5\\build_script_build-d6e71885dca539a5.exe" "/OPT:REF,NOICF" "/DEBUG" "/NATVIS:C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\bach_ar\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis"
  = note: LINK : fatal error LNK1181: Eingabedatei "kernel32.lib" kann nicht geöffnet werden.


error: could not compile `windows_x86_64_msvc` (build script) due to previous error
error: failed to compile `anstyle-query v1.0.0`, intermediate artifacts can be found at `C:\Users\bach_ar\AppData\Local\Temp\cargo-installIomCyX`

Sorry, Windows compilation is hard...

[max-sixty]

Oh no! That's not fun.

I think let's leave this open and see if we can get other reports. It's possibly an upstream config issue. I did some work to simplify the API so we didn't have to pass colors through the whole chain of commands — maybe we need to go back to the old way — but it'd be prudent to see some more reports before that.

If anyone needs an escape hatch in the meantime, I'm sure there are python libraries that would strip the codes — or this SO answer has a short regex for stripping them: (I realize this is not ideal...)

ansi_escape_8bit = re.compile(
    br'(?:\x1B[@-Z\\-_]|[\x80-\x9A\x9C-\x9F]|(?:\x1B\[|\x9B)[0-?]*[ -/]*[@-~])'
)
result = ansi_escape_8bit.sub(b'', somebytesvalue)

I'll change the subject of this too, if that's OK

[ArneBachmannDLR]

Sure, no worries