PyCall.jl icon indicating copy to clipboard operation
PyCall.jl copied to clipboard

Python's `sys.executable` points to Julia's binary on Windows

Open pedromxavier opened this issue 3 years ago • 6 comments

When Python's sys module is loaded via pyimport("sys") its executable returns Julia's path instead of Python's. The same doesn't happen on Linux.

To reproduce this issue (Windows 10, Julia 1.7.0, PyCall 1.93.0):

julia> using PyCall

julia> sys = pyimport("sys")
PyObject <module 'sys' (built-in)>

julia> sys.executable
"C:\\Users\\User\\AppData\\Local\\Programs\\Julia-1.7.0\\bin\\julia.exe"

pedromxavier avatar Jan 20 '22 02:01 pedromxavier

Isn't that correct? We are not running python.exe.

stevengj avatar Jan 20 '22 16:01 stevengj

Oh, I see, the definition of sys.executable in the Python docs is

A string giving the absolute path of the executable binary for the Python interpreter, on systems where this makes sense. If Python is unable to retrieve the real path to its executable, sys.executable will be an empty string or None.

(emphasis added)

See also these discussions about sys.executable for embedded python:

  • https://mail.python.org/archives/list/[email protected]/thread/O66N56PB4U6AGICGBSRFD2OWA5JWMFC6/
  • https://github.com/davidhalter/jedi-vim/issues/870

It seems like this happens a lot for programs embedding Python. In our case, PyCall.python is the path of the Python interpreter corresponding to the libpython we are linking to, so we should be able to set sys.executable to this.

stevengj avatar Jan 20 '22 16:01 stevengj

Very interesting definitions. On Linux the behavior is the following:

julia> using PyCall

julia> sys = pyimport("sys")
PyObject <module 'sys' (built-in)>

julia> sys.executable
"/usr/bin/python3"

as ,IMO, was expected. Apparently, there is no clear correspondence between PyCall.python and pyimport("sys").executable. This brings some inconsistency across platforms, getting even more unpredictable outcomes when PyCall.conda == true, which is the main point of #958.

pedromxavier avatar Jan 20 '22 17:01 pedromxavier

It looks like sys.executable is set from the configuration, and apparently we can use this API to change it when we initialize Python by using Py_InitializeFromConfig.

stevengj avatar Jan 21 '22 13:01 stevengj