CreateVirtualEnvironment.cmake fails with pyenv python shims

[astarche]

We've had several people (@cumitche , @phindman ) trying to contribute to grpc-device that have been blocked on an error from cmake:

CMake Error at C:/Program Files/CMake/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python3 (missing: Python3_EXECUTABLE Interpreter)



     Reason given by package:
          Interpreter: Cannot use the interpreter "C:/Users/username/.pyenv/pyenv-win/shims/python3"

I believe this is a compatibility issue between pyenv and the cmake python discovery code. There are several instances of "Cannot use the interpreter" in the cmake code that are all caused by a python one-liner failing with a given python. If we can figure out which one fails and what the error is, we can make some progress.

Note that pyenv is a recommended tool at NI, so this is a significant barrier to contributions.

There are 2 possible hack/local workarounds:

1. Install a system python (if possible, this may cause other issues) so that cmake will find it instead of the pyenv shim.
2. (Unconfirmed) Remove the call to find_package(Python3 REQUIRED COMPONENTS Interpreter) and replace with something like `set(Python3_EXECUTABLE, /path/to/pyenv/shims).

There is some related discussion in pyenv/pyenv#310. That looks like a different issue superficially, but maybe it's a different symptom of the same root cause.

AB#2181339

[bkeryan]

Note that pyenv and pyenv-win are separate projects. Here is a related pyenv-win issue: https://github.com/pyenv-win/pyenv-win/issues/352

[DavidCurtiss]

Another local workaround: create and use a virtual env.

[astarche]

Another local workaround: create and use a virtual env.

If that solves the Cannot use the interpreter issue, it sounds like a good official fix/workaround to me. Since our cmake code (when it works) creates the venv, we could probably tweak it to use/activate the existing one if it's already there. And you'd only need to create/activate it manually on the first build.

[kamathba]

I had a different issue with CreateVirtualEnvironment.make on Ubuntu 20.04 that I solved by manually creating a virtual environment and installing dependencies to it (and commenting out the call in CMakeLists.txt / defining PYTHON_EXE):

[  1%] Generating venv/environment.txt
/usr/bin/python3.8 -m venv --without-pip /home/bkamath/.dev/grpc-device/build/venv
venv/bin/python -m ensurepip --upgrade --default-pip
/home/bkamath/.dev/grpc-device/build/venv/bin/python: No module named ensurepip
make[2]: *** [CMakeFiles/virtual_environment.dir/build.make:62: venv/environment.txt] Error 1

[maxxboehme]

I am using pyenv 2.64.11 on Windows and haven't seen this issue. From the discussion when someone is experiencing this issue it appears that manually creating a venv and then activating that before building appears to work around the issue. Going to close this out until someone has an active instance of this issue and can debug into it further.