PyCall.jl
PyCall.jl copied to clipboard
JuliaPro installation error spawning "Temp\python.exe"
Hello,
I'm having the following error:
julia> using PyCall
[ Info: Precompiling PyCall [438e738f-606a-5dbb-bf0a-cddfbfd45ab0]
ERROR: LoadError: LoadError: PyCall not properly installed. Please run Pkg.build("PyCall")
Stacktrace:
[1] error(::String) at .\error.jl:33
[2] top-level scope at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\src\startup.jl:44 [inlined]
[3] top-level scope at .\none:0
[4] include at .\boot.jl:326 [inlined]
[5] include_relative(::Module, ::String) at .\loading.jl:1038
[6] include at .\sysimg.jl:29 [inlined]
[7] include(::String) at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\src\PyCall.jl:1
[8] top-level scope at none:0
[9] include at .\boot.jl:326 [inlined]
[10] include_relative(::Module, ::String) at .\loading.jl:1038
[11] include(::Module, ::String) at .\sysimg.jl:29
[12] top-level scope at none:2
[13] eval at .\boot.jl:328 [inlined]
[14] eval(::Expr) at .\client.jl:404
[15] top-level scope at .\none:3
in expression starting at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\src\startup.jl:41
in expression starting at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\src\PyCall.jl:41
ERROR: Failed to precompile PyCall [438e738f-606a-5dbb-bf0a-cddfbfd45ab0] to C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\compiled\v1.1\PyCall\GkzkC.ji.
Stacktrace:
[1] compilecache(::Base.PkgId, ::String) at .\loading.jl:1197
[2] _require(::Base.PkgId) at .\loading.jl:960
[3] require(::Base.PkgId) at .\loading.jl:858
[4] require(::Module, ::Symbol) at .\loading.jl:853
And I have this information:
julia> Pkg.build("PyCall")
Building Conda ─→ `C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\Conda\CpuvI\deps\build.log`
Building PyCall → `C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\deps\build.log`
┌ Error: Error building `PyCall`:
│ ┌ Info: Using the Python distribution in the Conda package by default.
│ └ To use a different Python version, set ENV["PYTHON"]="pythoncommand" and re-run Pkg.build("PyCall").
│ ERROR: LoadError: IOError: could not spawn setenv(`'C:\Users\PAOLOP~1\AppData\Local\Temp\python.exe' 'C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\deps\find_libpython.py' --list-all`,["USERDOMAIN_ROAMINGPROFILE=SAMSUNG", "DISABLE_FALLBACK=true", "HOMEPATH=\\Users\\Paolo Ponce", "ProgramData=C:\\ProgramData", "ProgramW6432=C:\\Program Files", "PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL", "NODE_ENV=production", "SESSIONNAME=Console", "APPDATA=C:\\Users\\Paolo Ponce\\AppData\\Roaming", "PUBLIC=C:\\Users\\Public", "USERDOMAIN=SAMSUNG", "OS=Windows_NT", "PROCESSOR_REVISION=3a09", "TMP=C:\\Users\\PAOLOP~1\\AppData\\Local\\Temp", "JULIA_EDITOR=\"C:\\JuliaPro-1.1.0.1\\app-1.36.0\\atom.exe\" -a", "ALLUSERSPROFILE=C:\\ProgramData", "Path=C:\\JuliaPro-1.1.0.1\\Julia-1.1.0\\share\\julia\\stdlib\\v1.1\\MbedTLS;C:\\JuliaPro-1.1.0.1\\Julia-1.1.0\\share\\julia\\stdlib\\v1.1\\MbedTLS\\deps\\usr\\bin;C:\\JuliaPro-1.1.0.1\\Julia-1.1.0\\share\\julia\\stdlib\\v1.1\\MbedTLS;C:\\JuliaPro-1.1.0.1\\Julia-1.1.0\\share\\julia\\stdlib\\v1.1\\MbedTLS\\deps\\usr\\bin;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\MiKTeX 2.9\\miktex\\bin\\x64\\;C:\\Program Files (x86)\\LyX 2.2\\Perl\\bin;C:\\Program Files\\MATLAB\\R2018a\\runtime\\win64;C:\\Program Files\\MATLAB\\R2018a\\bin;C:\\WINDOWS\\System32\\OpenSSH\\;\";C:\\cygwin64\\bin;C:\\cygwin64\\usr\\bin;C:\\cygwin64\\usr\\local\\bin;C:\\cygwin64\\lib;C:\\cygwin64\\usr\\lib\";C:\\cygwin64\\bin;C:\\cygwin64\\usr\\bin;C:\\cygwin64\\usr\\local\\bin;C:\\cygwin64\\lib;C:\\cygwin64\\usr\\lib;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\MiKTeX 2.9\\miktex\\bin\\x64\\;C:\\Program Files (x86)\\LyX 2.2\\Perl\\bin;C:\\Program Files\\MATLAB\\R2018a\\runtime\\win64;C:\\Program Files\\MATLAB\\R2018a\\bin;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Users\\Paolo Ponce\\AppData\\Local\\Microsoft\\WindowsApps;;C:\\Anaconda3\\Scripts;C:\\Anaconda3", "COMPUTERNAME=SAMSUNG", "USERNAME=Paolo Ponce", "CommonProgramFiles(x86)=C:\\Program Files (x86)\\Common Files", "CommonProgramFiles=C:\\Program Files\\Common Files", "USERPROFILE=C:\\Users\\Paolo Ponce", "GOOGLE_API_KEY=AIzaSyAQfxPJiounkhOjODEO5ZieffeBv6yft2Q", "PSModulePath=C:\\Program Files\\WindowsPowerShell\\Modules;E:\\Documents\\WindowsPowerShell\\Modules;C:\\Program Files (x86)\\WindowsPowerShell\\Modules;C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\Modules", "PROCESSOR_LEVEL=6", "JULIA_PKG_TOKEN_PATH=C:\\Users\\Paolo Ponce\\.juliapro\\token.toml", "=C:=C:\\Users\\Paolo Ponce\\.juliapro\\JuliaPro_v1.1.0.1\\packages\\PyCall\\ttONZ\\deps", "TERM=xterm-256color", "SystemDrive=C:", "HOMEDRIVE=C:", "TEMP=C:\\Users\\PAOLOP~1\\AppData\\Local\\Temp", "PYTHONIOENCODING=UTF-8", "ESET_OPTIONS=
", "NODE_PATH=C:\\JuliaPro-1.1.0.1\\app-1.36.0\\resources\\app.asar\\exports", "LOCALAPPDATA=C:\\Users\\Paolo Ponce\\AppData\\Local", "PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 58 Stepping 9, GenuineIntel", "DriverData=C:\\Windows\\System32\\Drivers\\DriverData", "NUMBER_OF_PROCESSORS=4", "ComSpec=C:\\WINDOWS\\system32\\cmd.exe", "PROMPT=\$P\$G", "SystemRoot=C:\\WINDOWS", "JULIA_PKG_SERVER=https://pkg.juliacomputing.com/", "OneDrive=C:\\Users\\Paolo Ponce\\OneDrive", "ProgramFiles(x86)=C:\\Program Files (x86)", "JULIA_NUM_THREADS=2", "PSExecutionPolicyPreference=Bypass", "JUNORC_PATH=C:\\JuliaPro-1.1.0.1\\.atom", "LOGONSERVER=\\\\SAMSUNG", "windir=C:\\WINDOWS", "ATOM_HOME=C:\\JuliaPro-1.1.0.1\\.atom", "JULIA_LOAD_PATH=@;C:\\Users\\PAOLOP~1\\AppData\\Local\\Temp\\jl_30A9.tmp", "CommonProgramW6432=C:\\Program Files\\Common Files", "ProgramFiles=C:\\Program Files", "PROCESSOR_ARCHITECTURE=AMD64", "OPENBLAS_MAIN_FREE=1"]): no such file or directory (ENOENT)
│ Stacktrace:
│ [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:400
│ [2] #505 at .\process.jl:413 [inlined]
│ [3] setup_stdios(::getfield(Base, Symbol("##505#506")){Cmd}, ::Array{Any,1}) at .\process.jl:497
│ [4] _spawn at .\process.jl:412 [inlined]
│ [5] #eachline#511(::Bool, ::Function, ::Cmd) at .\process.jl:618
│ [6] eachline at .\process.jl:617 [inlined]
│ [7] #readlines#288 at .\io.jl:414 [inlined]
│ [8] readlines at .\io.jl:414 [inlined]
│ [9] exec_find_libpython at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\deps\buildutils.jl:28 [inlined]
│ [10] #find_libpython#3(::typeof(Libdl.dlopen), ::Function, ::String) at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\deps\buildutils.jl:43
│ [11] find_libpython(::String) at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\deps\buildutils.jl:41
│ [12] top-level scope at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\deps\build.jl:87
│ [13] include at .\boot.jl:326 [inlined]
│ [14] include_relative(::Module, ::String) at .\loading.jl:1038
│ [15] include(::Module, ::String) at .\sysimg.jl:29
│ [16] include(::String) at .\client.jl:403
│ [17] top-level scope at none:0
│ in expression starting at C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\PyCall\ttONZ\deps\build.jl:43
└ @ Pkg.Operations C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\Operations.jl:1136
My Julia info:
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i3-3120M CPU @ 2.50GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, ivybridge)
Environment:
JULIA_EDITOR = "C:\JuliaPro-1.1.0.1\app-1.36.0\atom.exe" -a
JULIA_NUM_THREADS = 2
JULIA_PKG_SERVER = https://pkg.juliacomputing.com/
JULIA_PKG_TOKEN_PATH = C:\Users\Paolo Ponce\.juliapro\token.toml
It's strange that PyCall is trying to use C:\Users\PAOLOP~1\AppData\Local\Temp\python.exe while this Python is supposed to be from conda.
- What is the value of
using Conda; Conda.PYTHONDIR? You may need to runPkg.add("Conda")first. - Did you set
CONDA_JL_HOMEto change the location of Conda installation? - Your log says
Building Conda ─→ `C:\Users\Paolo Ponce\.juliapro\JuliaPro_v1.1.0.1\packages\Conda\CpuvI\deps\build.log`. What is in thisbuild.logfile?
I've got the same error. Also on the JuliaPro1.1.0 version... Conda.PYHTONDIR is the same user\AppData\Local\Temp\ directory as for him. I didn't set CONDA_JL_HOME to anything. Only thing that might have happened is that since JuliaPro installs everything into a manually selected directory it might redirect julia paths in an incompatible manner. the build.log file contains exactly that what follows the corresponding line in the output. (No new information)
Output of
using Conda
@info(
"Conda.jl configuration",
Conda.ROOTENV,
Conda.MINICONDA_VERSION,
Conda.deps_file,
Conda.PREFIX,
Conda.BINDIR,
Conda.LIBDIR,
Conda.SCRIPTDIR,
Conda.PYTHONDIR,
Conda.conda,
Conda.CONDARC,
)
Conda.runconda(`info`)
may be useful.
Definitively looks like some path has been set to /Temp/ ┌ Info: Conda.jl configuration │ Conda.ROOTENV = "C:\Users\necka\AppData\Local\Temp\" │ Conda.MINICONDA_VERSION = "3" │ Conda.deps_file = "C:\Users\necka\.juliapro\JuliaPro_v1.1.0.1\packages\Conda\CpuvI\src\..\deps\deps.jl" │ Conda.PREFIX = "C:\Users\necka\AppData\Local\Temp\" │ Conda.BINDIR = "C:\Users\necka\AppData\Local\Temp\Library\bin" │ Conda.LIBDIR = "C:\Users\necka\AppData\Local\Temp\Library\bin" │ Conda.SCRIPTDIR = "C:\Users\necka\AppData\Local\Temp\Scripts" │ Conda.PYTHONDIR = "C:\Users\necka\AppData\Local\Temp\" │ Conda.conda = "C:\Users\necka\AppData\Local\Temp\Scripts\conda.exe" └ Conda.CONDARC = "C:\Users\necka\AppData\Local\Temp\condarc-julia.yml"
And output from Conda.runconda('info')
[ Info: Downloading miniconda installer ...
[ Info: Installing miniconda ...
[ Info: Running conda config --add channels defaults --file 'C:\Users\necka\AppData\Local\Temp\condarc-julia.yml' --force in root
environment
Warning: 'defaults' already in 'channels' list, moving to the top
[ Info: Running conda update -y conda in root environment
Solving environment: done
Package Plan
environment location: C:\Users\necka\AppData\Local\Temp\
added / updated specs: - conda
The following packages will be downloaded:
package | build
---------------------------|-----------------
cryptography-2.6.1 | py36h7a1dbc1_0 563 KB
openssl-1.1.1b | he774522_1 5.7 MB
vs2015_runtime-14.15.26706 | h3a45250_4 2.4 MB
ca-certificates-2019.1.23 | 0 158 KB
certifi-2019.3.9 | py36_0 156 KB
conda-4.6.14 | py36_0 2.1 MB
vc-14.1 | h0510ff6_4 6 KB
------------------------------------------------------------
Total: 11.1 MB
The following packages will be UPDATED:
ca-certificates: 2018.03.07-0 --> 2019.1.23-0
certifi: 2018.4.16-py36_0 --> 2019.3.9-py36_0
conda: 4.5.4-py36_0 --> 4.6.14-py36_0
cryptography: 2.2.2-py36hfa6e2cd_0 --> 2.6.1-py36h7a1dbc1_0
openssl: 1.0.2o-h8ea7d77_0 --> 1.1.1b-he774522_1
vc: 14-h0510ff6_3 --> 14.1-h0510ff6_4
vs2015_runtime: 14.0.25123-3 --> 14.15.26706-h3a45250_4
Downloading and Extracting Packages
cryptography-2.6.1 | 563 KB | ########################################################################################## | 100%
openssl-1.1.1b | 5.7 MB | ########################################################################################## | 100%
vs2015_runtime-14.15 | 2.4 MB | ########################################################################################## | 100%
ca-certificates-2019 | 158 KB | ########################################################################################## | 100%
certifi-2019.3.9 | 156 KB | ########################################################################################## | 100%
conda-4.6.14 | 2.1 MB | ########################################################################################## | 100%
vc-14.1 | 6 KB | ########################################################################################## | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
[ Info: Running conda info in root environment
active environment : base
active env location : C:\Users\necka\AppData\Local\Temp\
user config file : C:\Users\necka\.condarc
populated config files : C:\Users\necka\AppData\Local\Temp\condarc-julia.yml conda version : 4.6.14 conda-build version : not installed python version : 3.6.5.final.0 base environment : C:\Users\necka\AppData\Local\Temp (writable) channel URLs : https://repo.anaconda.com/pkgs/main/win-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/win-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/win-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/msys2/win-64 https://repo.anaconda.com/pkgs/msys2/noarch package cache : C:\Users\necka\AppData\Local\Temp\pkgs C:\Users\necka.conda\pkgs C:\Users\necka\AppData\Local\conda\conda\pkgs envs directories : C:\Users\necka\AppData\Local\Temp\envs C:\Users\necka.conda\envs C:\Users\necka\AppData\Local\conda\conda\envs platform : win-64 user-agent : conda/4.6.14 requests/2.18.4 CPython/3.6.5 Windows/10 Windows/10.0.17134 administrator : False netrc file : None offline mode : False
What is your DEPOT_PATH (a variable available in Julia REPL)?
If that does not have any suspicious paths, maybe removing C:\Users\necka\.juliapro\JuliaPro_v1.1.0.1\packages\Conda\CpuvI\src\..\deps\deps.jl and then re-running Pkg.build("Conda") can fix it.
julia> ENV["DEPOT_PATH"]
ERROR: KeyError: key "DEPOT_PATH" not found
Stacktrace:
[1] (::getfield(Base, Symbol("##434#435")))(::String) at .\env.jl:76
[2] access_env(::getfield(Base, Symbol("##434#435")), ::String) at .\env.jl:14
[3] getindex(::Base.EnvDict, ::String) at .\env.jl:76
[4] top-level scope at none:0
DEPOT_PATH is not an environment variable, it is just an ordinary Julia variable.
Note that the default Conda rootenv is not stored in the package directory, it is stored in ~/.julia/conda/deps.jl (so that it persists across upgrades). To remove it and reinstall Conda properly, do:
import Conda
rm(Conda.ROOTENV, recursive=true)
rm(abspath(first(DEPOT_PATH), "conda", "deps.jl"))
and re-build Conda. (I'm also assuming that the CONDA_JL_HOME environment variable is not set, as otherwise that could change the rootenv.)
Hello, I solved the problem using this: http://www.math.brown.edu/~sswatson/classes/data1010/setup.html
Can you please reopen the issue since the problem is not resolved (and other people might encounter this problem aswell).
Is @PaoloPonce is willing to continue debugging his Conda-based configuration? Otherwise we may have to close as nonreproducible.
I'll continue it once I get to my computer.
@stevengj Maybe it's unsafe to remove Conda.ROOTENV as it's C:\Users\...\AppData\Local\Temp\ here? Some applications may be storing some temporary data there and deleting it could potentially corrupt their state? (I'm assuming this directory is somewhat equivalent to /tmp for Linux.)
julia> DEPOT_PATH
3-element Array{String,1}:
"C:\\Users\\necka\\.julia"
"C:\\Users\\necka\\AppData\\Local\\Julia-1.0.2\\local\\share\\julia"
"C:\\Users\\necka\\AppData\\Local\\Julia-1.0.2\\share\\julia"
Note that the default Conda rootenv is not stored in the package directory, it is stored in
~/.julia/conda/deps.jl(so that it persists across upgrades). To remove it and reinstall Conda properly, do:import Conda rm(Conda.ROOTENV, recursive=true) rm(abspath(first(DEPOT_PATH), "conda", "deps.jl"))and re-build Conda. (I'm also assuming that the
CONDA_JL_HOMEenvironment variable is not set, as otherwise that could change the rootenv.)
Did remove everything in /Temp/ that looked like it was from python/conda/julia and removed that deps.jl Then I did rm Conda, add Conda and build Conda. Conda.ROOTENV still points to /Temp/
and calling add PyCall and build PyCall again ends in the same error where we started
Did you remove "C:\Users\necka\.julia\conda\deps.jl"? That's where the ROOTENV setting is saved between builds. Is the CONDA_JL_HOME environment variable unset?
Removing like everything that contains 'conda' in its name recursively in the .juliapro directory finally solved it. :D Thanks for support, though I would've been quite curious where the actual problem came from. Shall we let it open for people that encounter the same problem?
Since we don't know what your actual problem was, and it is no longer reproducible, I'm closing this. We can reopen if someone reproduces the same problem.
Re-opening this issue, since it still seems to be happening to JuliaPro users. cc @aviks @StefanKarpinski @bmharsha
Meanwhile, a workaround is:
import Pkg
rm(joinpath(homedir(), ".julia", "conda"), recursive=true)
Pkg.build("PyCall")
@ViralBShah, does JuliaPro on Windows come prepackaged with a .julia\conda\deps.jl file that has an invalid ROOTENV path or something?
I'll investigate this in detail, but one key feature is that JuliaPro does not use .julia as it's depot path. It uses .juliapro/JuliPro_vx.x.x.x. JuliaPro does not write anything in .julia.
@aviks, both Conda.jl and PyCall.jl use first(DEPOT_PATH). In particular, key preferences/paths are stored in joinpath(first(DEPOT_PATH), "prefs", "PyCall") and abspath(first(DEPOT_PATH), "conda", "deps.jl")
Yeah, and that should point to the .juliapro directory. I tried to build PyCall on a juliapro/win10 environment, and it seems to have done that successfully. Below is the end of they PyCall install log.
[ Info: PyCall is using C:\Users\avik\.juliapro\JuliaPro_v1.0.5-2\Conda_env\python.exe (Python 3.6.5) at C:\Users\avik\.juliapro\JuliaPro_v1.0.5-2\Conda_env\python.exe, libpython = C:\Users\avik\.juliapro\JuliaPro_v1.0.5-2\Conda_env\python36.dll
[ Info: C:\Users\avik\.juliapro\JuliaPro_v1.0.5-2\packages\PyCall\ttONZ\deps\deps.jl has been updated
[ Info: C:\Users\avik\.juliapro\JuliaPro_v1.0.5-2\prefs\PyCall has been updated
Weird. Someone reported this problem again just a couple weeks ago: https://discourse.julialang.org/t/error-building-pycall/16270/9
There may be a python.exe on the path in newer Win10 machines, one that is not a real python interpreter. Could that possibly cause issues?
https://docs.microsoft.com/en-us/windows/python/faqs#why-does-running-pythonexe-open-the-microsoft-store
Hello, I was having same problem when I tried to install PyCall. (Pardon me if anything I say is misleading or stupid, I am very new). I tried the workaround in the thread, but it did not appear to change anything.
I browsed around my files for a while, and noticed that installing Conda from the Julia terminal did not actually build a python environment (no Python executable anywhere). I ended up fixing the issue by separately installing Anaconda, and redirecting to the anaconda environment.
ENV["PYTHON"] = "C:\\Users\\Dujek\\Anaconda3\\python.exe"
Pkg.build("PyCall")
Hope this helps someone!
There may be a python.exe on the path in newer Win10 machines, one that is not a real python interpreter. Could that possibly cause issues?
PyCall is supposed to use Conda.jl by default on Windows, unless you set the PYTHON environment variable.
What to do, it says
PyCall not properly installed. Please run Pkg.build("PyCall")
And I can not build PyCall