Update Windows builds to use ClangCL

[zanieb]

Adding the flag as described in https://github.com/python/cpython/pull/129907

ClangCL looks to provide a 20% runtime performance and build times were purportedly better as well.

This also would let us turn on the tail-calling interpreter for another 10% performance boost.

Waiting to test this until the new CPython 3.14 alpha is out.

[zanieb]

The latest benchmark is at https://github.com/python/cpython/pull/129907#issuecomment-2701933114 which shows some regressions, but an average 19% improvement.

[zanieb]

Interesting, it failed with

cpython>        (ClCompile target) ->
cpython>          C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.43.34808\include\vcruntime.h(188,30): error : typedef redefinition with different types ('unsigned long long' vs 'unsigned int') [C:\Users\RUNNER~1\AppData\Local\Temp\python-build-73gmp1o2\Python-3.14.0a6\PCbuild\pythoncore.vcxproj]
cpython>          C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\Llvm\lib\clang\19\include\__stddef_ptrdiff_t.h(18,26): error : typedef redefinition with different types ('int' vs 'long long') [C:\Users\RUNNER~1\AppData\Local\Temp\python-build-73gmp1o2\Python-3.14.0a6\PCbuild\pythoncore.vcxproj]
cpython>          C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.43.34808\include\vcruntime.h(188,30): error : typedef redefinition with different types ('unsigned long long' vs 'unsigned int') [C:\Users\RUNNER~1\AppData\Local\Temp\python-build-73gmp1o2\Python-3.14.0a6\PCbuild\pythoncore.vcxproj]
cpython>          C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\Llvm\lib\clang\19\include\__stddef_ptrdiff_t.h(18,26): error : typedef redefinition with different types ('int' vs 'long long') [C:\Users\RUNNER~1\AppData\Local\Temp\python-build-73gmp1o2\Python-3.14.0a6\PCbuild\pythoncore.vcxproj]

[chris-eibl]

AFAIR this means it is compiling in 323bit mode. Try adding -m64.

See https://github.com/python/cpython/pull/129907/commits/263870dd319e79170f42fc5e05beb3879effff7b.

Interesting, according to https://github.com/python/cpython/commit/d8a1cf469e40b538a84e583ac1d5d34418ed9a6a this should be part of v3.14.0a6

[chris-eibl]

But I see it is passed e.g. here:

-flto=thin -m64 -Wno-deprecated-non-prototype -Wno-unused-label -Wno-pointer-sign -Wno-incompatible-pointer-types-discards-qualifiers -Wno-unused-function  "..\Modules\_heapqmodule.c"

but not here

 /Z7 /nologo /W3 /WX- /diagnostics:column /O2 /Oi /D HACL_CAN_COMPILE_VEC256 /D _Py_HAVE_ZLIB /D _USRDLL /D Py_BUILD_CORE /D Py_BUILD_CORE_BUILTIN /D Py_ENABLE_SHARED /D "MS_DLL_ID=\"3.14\"" /D _Py_USING_PGO=1 /D WIN32 /D "PY3_DLLNAME=L\"python3\"" /D _WIN64 /D NDEBUG /D PyStats /D _WINDLL /GF /MD /GS /Gy /fp:precise /Fo"C:\Users\RUNNER~1\AppData\Local\Temp\python-build-2aa_wvqi\Python-3.14.0a6\PCbuild\obj\314amd64_PGInstrument\pythoncore\\" /Gd /TC /arch:AVX2 ..\Modules\_hacl\Hacl_Hash_Blake2b_Simd256.c

But in the line above for \Hacl_Hash_Blake2s.c all the clang stuff is present like for _heapqmodule.c above :-o

[chris-eibl]

Try adding "/p:PreferredToolArchitecture=x64". Before https://github.com/python/cpython/commit/263870dd319e79170f42fc5e05beb3879effff7b that was the trick that did it for me. Maybe it is still needed, although I back then did not need it anymore. That was a back and forth.

I will re-investigate ...

[zanieb]

We also do some non-standard things (like, we hack the project files), so it's possible it's a problem there.

[chris-eibl]

Interestingly, I can reproduce on cpython main and alpha6. I cannot build without "/p:PreferredToolArchitecture=x64" anymore, too. Weird, but I have an idea.

Need some sleep though now :)

[zanieb]

Cool that gets us to

cpython> copying C:\Users\RUNNER~1\AppData\Local\Temp\python-build-hctl79hu\Python-3.14.0a6\PCbuild\amd64\python314.exp
Traceback (most recent call last):
  File "C:\a\python-build-standalone\python-build-standalone\cpython-windows\build.py", line 1816, in <module>
    sys.exit(main())
             ^^^^^^
  File "C:\a\python-build-standalone\python-build-standalone\cpython-windows\build.py", line 1790, in main
    tar_path = build_cpython(
               ^^^^^^^^^^^^^^
  File "C:\a\python-build-standalone\python-build-standalone\cpython-windows\build.py", line 1524, in build_cpython
    build_info = collect_python_build_artifacts(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\a\python-build-standalone\python-build-standalone\cpython-windows\build.py", line 1079, in collect_python_build_artifacts
    shutil.copyfile(source, dest)
  File "C:\hostedtoolcache\windows\Python\3.12.9\x64\Lib\shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\python-build-hctl79hu\\Python-3.14.0a6\\PCbuild\\amd64\\python314.exp'

which is from the copy at https://github.com/astral-sh/python-build-standalone/blob/093564335adaf1af9beab9cd6d771b4aa94a3dba/cpython-windows/build.py#L1072-L1079

I presume this file is not produced upstream as well? I don't know if it's of consequence — I'm woefully lacking in knowledge of .exp files / Windows dynamic linking details.

[chris-eibl]

Ok, I've dug deeper this time - please see https://github.com/python/cpython/issues/131473 for details.

Regarding the *.exp file: most probably the gold linker does not write that, I did not find anything in the net, yet. Don't know whether this is a clang-cl compatibility issue or a wrong switch, etc, in the cPython build.

[zanieb]

A nice improvement in build times

• i686-pc-windows-msvc / 3.14 / pgo 29m -> 22m
• x86_64-pc-windows-msvc / 3.14 / pgo 26m -> 18m

[Fidget-Spinner]

Tail calling for Windows clangcl has been merged https://github.com/python/cpython/pull/130040/files#diff-4295039788386c2986b811f2a6b59cf088be59822b24b4f1243c9e9b184f65fbR109