"Could not locate executable cl.exe" when building package for Python 3.6 on Windows

[seibert]

We discovered that building Numba using our usual development recipe:

https://github.com/numba/numba/tree/master/buildscripts/condarecipe.local

is failing with conda-build 3.18.x on our Windows build server only when we try to build Python 3.6 or 3.7, but not 2.7 We get this error at the end of the build process:

cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I%PREFIX%\include -I%PREFIX%\include -I%PREFIX%\Library\include -I%PREF     IX%\Library\include /Tcnumba/_dynfuncmod.c /Fobuild\temp.win-amd64-3.6\Release\numba/_dynfuncmod.obj -g
Could not locate executable cl.exe
Executable cl.exe does not exist

Stepping back through released versions of conda build, we have determined that with conda build 3.17.8:

• works: conda build --no-test --python=2.7 --numpy=1.11 buildscripts/condarecipe.local/
• works: conda build --no-test --python=3.6 --numpy=1.11 buildscripts/condarecipe.local/

conda-build 3.18.5:

• works: conda build --no-test --python=2.7 --numpy=1.11 buildscripts/condarecipe.local/
• doesn't work: conda build --no-test --python=3.6 --numpy=1.11 buildscripts/condarecipe.local/

In all cases, this is with conda 4.7.11 installed.

[seibert]

Edits to make it clear that I'm talking about Windows builds.

[jjhelmus]

I'm seeing similar behavior on AppVeyor when testing conda-package-handling, (build log). It looks like the logic that detects where Visual Studio is installed is failing and falling back to the "Enterprise" default which is not installed:

%SRC_DIR%>set "VSINSTALLDIR=" 
%SRC_DIR%>if exist "C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" (for /F "usebackq tokens=*" %i in (`"C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" -nologo -products * -version [15.0,16.0) -property installationPath`) do (set "VSINSTALLDIR=%i\" ) ) 
%SRC_DIR%>if not exist "" (set "VSINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\" ) 
%SRC_DIR%>if not exist "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\" (set "VSINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\" ) 
%SRC_DIR%>if not exist "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\" (set "VSINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\" ) 
%SRC_DIR%>if not exist "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\" (set "VSINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\" ) 
%SRC_DIR%>IF NOT "1" == "" (
set "INCLUDE=%PREFIX%\Library\include;%PREFIX%\Library\include;"  
 set "LIB=%PREFIX%\Library\lib;%PREFIX%\Library\lib;"  
 set "CMAKE_PREFIX_PATH=%PREFIX%\Library;" 
) 
%SRC_DIR%>pushd "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\"  
The system cannot find the path specified.
%SRC_DIR%>CALL "VC\Auxiliary\Build\vcvars64.bat"  
The system cannot find the path specified.
%SRC_DIR%>popd  
%SRC_DIR%>IF "" == "" (SET "CMAKE_GENERATOR=Visual Studio 15 2017 Win64"  ) 

This logic is present in the activation script of the vs2017_win-64 package.

@seibert do see similar issues in your logs? Specifically anything along the lines of: pushd "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\".

[msarahan]

VS 2017 recipe was reworked a bit this past week to use Microsoft's "vswhere" tool. @seibert if you could verify whether your problem is still present, it would be helpful.

[seibert]

Which conda-build do I need to install? Latest from conda-canary?

[msarahan]

no change to conda-build is necessary. It's just which compiler package gets pulled in at build time. The latest is version 19.16.27032.1

[jjhelmus]

The default compiler used when building a Python 3 package on Windows changed in #3517 from vs2015 to vs2017. This change is present in conda-build 3.18.3 and later.

If only VS 2015 is installed and not VS 2017 then a build would succeed using conda-build <3.18.3 but fail with conda-build >=3.18.3 as the vs2017_win-64 package will attempt to activate the VS 2017 tools not the VS 2015 ones.

The compilers used can be specified explicitly in a conda_build_config.yaml file in the recipe directory or other locations. The following specifies a matrix of three builds for Python 2.7, 3.6 and 3.7 with vs2008 being used for the Python 2.7 build and vs2015 for the 3.6 and 3.7 builds:

c_compiler:
  - vs2008
  - vs2015
  - vs2015
cxx_compiler:
  - vs2008
  - vs2015
  - vs2015
python:
  - 2.7
  - 3.6
  - 3.7
zip_keys:
  - c_compiler
  - cxx_compiler
  - python

[oleg-alexandrov]

Here getting the same issue. I have conda-build 3.22.0. In conda_build_config.yaml I see vs2017. Python version is 3.9.12. This is from a fresh install of miniconda for Windows.

[github-actions[bot]]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include: - What OS and version you reproduced the issue on - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!