Building on Windows from `cpt.py` fails with dozens of: `clang-tblgen.exe: Unknown command line argument ...`

[GavinRay97]

• [x] Checked for duplicates

Describe the bug

Trying to build with cpt.py --check-requirements && cpt.py --create-dev-env Debug --with-workdir=./cling-build/ eventually fails with:

Here's the section containing the actual CMake args and my MSVC version, full log uploaded at gist below:

• https://gist.github.com/GavinRay97/de585c5e3b71760457d5336bd878434b

+-----------------------------------------------------------------------------+
| Set Cling version                                                           |
+-----------------------------------------------------------------------------+
Version: 1.0~dev-668dfdc
Revision: 668dfdcc7ff8bc47ed9accb20b8639adcb7e80fb

...

+-----------------------------------------------------------------------------+
| Configure Cling with CMake C:\Users\rayga\Projects\tmp\cling\cling-build\cling-src -DLLVM_BUILD_TOOLS=Off -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=C:\Windows\Temp\cling-obj\  -DLLVM_TARGETS_TO_BUILD="host;NVPTX" |
+-----------------------------------------------------------------------------+
cmake.exe C:\Users\rayga\Projects\tmp\cling\cling-build\cling-src -DLLVM_BUILD_TOOLS=Off -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=C:\Windows\Temp\cling-obj\  -DLLVM_TARGETS_TO_BUILD="host;NVPTX"
-- Building for: Visual Studio 16 2019
CMake Warning at CMakeLists.txt:37 (message):
  Visual Studio generators use the x86 host compiler by default, even for
  64-bit targets.  This can result in linker instability and out of memory
  errors.  To use the 64-bit host compiler, pass -Thost=x64 on the CMake
  command line.


-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.21376.
-- The C compiler identification is MSVC 19.28.29913.0
-- The CXX compiler identification is MSVC 19.28.29913.0

To Reproduce

1. Download cpt.py from URL provided
2. Run cpt.py --check-requirements && cpt.py --create-dev-env Debug --with-workdir=./cling-build/ on Windows

[GavinRay97]

This appears to actually be an issue with MSVC -- not even -help command works. The entire CLI parsing is completely borked.

• [llvm] llvm-tblgen.exe fails when building LLVM - https://github.com/microsoft/vcpkg/issues/11558
• [llvm] fix llvm-tblgen build with MSVC v19.26 - https://github.com/microsoft/vcpkg/pull/11703

The fix was merged + backported into 10.x release branch, so no dice for Clang 9 it seems =( Working options are:

• MSVC (any version) w/ Clang +10
• MSVC prior to 14.26 (IE 14.25) w/ Clang 9

"LLVM's ManagedStatic.h fails with VS 2019 and clang-cl due to init-order-fiasco and constexpr confusion"

• https://bugs.llvm.org/show_bug.cgi?id=41367

Going to try to install MSVC from the Visual Studio 2017 Build Tools workload in the VS Installer and see if that does the trick. (I just have to figure out how to tell CMake/cpt.py to use that version instead)

[GavinRay97]

For historical purposes/anyone else facing issues, I've managed to compile it successfully with:

# Source ENV vars for cl.exe and build environment
set VSCMD_DEBUG=2
"C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\Tools\VsDevCmd.bat" -arch=amd64 -host_arch=x64

# Run the build -- EDITTED WITH UPDATE BELOW
cpt.py --check-requirements && cpt.py --verbose --create-dev-env Debug --with-workdir=./cling-build/ --with-cmake-flags="cmake -G 'Visual Studio 15 2017' -A x64 -Thost=x64"

Though linking fails with:

  clingUtils.vcxproj -> C:\Users\rayga\Projects\tmp\cling\cling-build\builddir\Debug\lib\clingUtils.lib
  Building Custom Rule C:/Users/rayga/Projects/tmp/cling/cling-build/cling-src/tools/cling/tools/driver/CMakeLists.txt
  cling.cpp

     Creating library C:/Users/rayga/Projects/tmp/cling/cling-build/builddir/Debug/lib/cling.lib and object C:/Users/rayga/Projects/tmp/cling/
  cling-build/builddir/Debug/lib/cling.exp

LINK : fatal error LNK1210: exceeded internal ILK size limit; link with /INCREMENTAL:NO [C:\Users\rayga\Projects\tmp\cling\cling-build\builddir\tools\cling\tools\driver\cling.vcxproj]
subprocess.CalledProcessError: Command 'cmake.exe --build . --target cling --config Debug' returned non-zero exit status 1

UPDATE: Solved by passing -Thost=x64 to --with-cmake-flags= There's one final thing -- during building the debug/PDB support, it's missing the ATL header, so you'll need to install this workload from the VS Installer as well if you want this functionality:

cling-build\cling-src\include\llvm\debuginfo\pdb\dia\DIASupport.h(25): fatal error C1083:
Cannot open include file: 'atlbase.h': No such file or directory

cling runs without it though

[Axel-Naumann]

@reikdas would you mind having a look?

[PavelKotov1]

Hello, dear Friends! i was trying to build llvm.sln in vs 2019 and received numerous errors similar to one seen above (regarding tblgen.exe file). Here is just an examle:

95>Building LanaiGenAsmMatcher.inc...
95>'..\..\..\Debug\bin\llvm-tblgen.exe' is not recognized as an internal or external command,
95>operable program or batch file.
95>C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for 'K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenAsmMatcher.inc.rule;K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenAsmWriter.inc.rule;K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenCallingConv.inc.rule;K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenDAGISel.inc.rule;K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenDisassemblerTables.inc.rule;K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenInstrInfo.inc.rule;K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenMCCodeEmitter.inc.rule;K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenRegisterInfo.inc.rule;K:\llvm-project\build\CMakeFiles\c911c0e06961fc0460d97a5b7cb1a486\LanaiGenSubtargetInfo.inc.rule;K:\llvm-project\build\CMakeFiles\258802e33682ca55d54de3439ea6f272\LanaiCommonTableGen.rule' exited with code 9009.
95>Done building project "LanaiCommonTableGen.vcxproj" -- FAILED.
102>------ Build started: Project: InstallNameToolOptsTableGen, Configuration: Debug x64 ------
97>Building Custom Rule K:/llvm-project/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
96>Building NVPTXGenAsmWriter.inc...
96>'..\..\..\Debug\bin\llvm-tblgen.exe' is not recognized as an internal or external command,
96>operable program or batch file.

etc. I am very new to llvm and just learning my ropes and trying to find my way around, so please point me in the right direction! Thanks in advance.

[ferdymercury]

Could you retry with latest master and reopen if it fails?