pyzmq
pyzmq copied to clipboard
zeromq
cannot import name 'constants' from partially initialized module 'zmq.backend.cython'
Have installed Python 3.9.0 on Windows 10 64 bit and installed jupyter and dependencies using pip. The installation goes fine, but when running "jupyter notebook" from the command line, I get the following error which seems to be caused by pyzmq. The installed pyzmq is 20.0.0:
Traceback (most recent call last):
File "c:\program files\python\python-3.9.0\lib\runpy.py", line 197, in run_module_as_main
return run_code(code, main_globals, None,
File "c:\program files\python\python-3.9.0\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Program Files\Python\Python-3.9.0\Scripts\jupyter-notebook.EXE_main.py", line 4, in
I am also having this issue, but only when using the debug interpreter:
> D:\Python38\python_d.exe
Python 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:37:42) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import zmq
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Python38\lib\site-packages\zmq\__init__.py", line 55, in <module>
from zmq import backend
File "D:\Python38\lib\site-packages\zmq\backend\__init__.py", line 40, in <module>
reraise(*exc_info)
File "D:\Python38\lib\site-packages\zmq\utils\sixcerpt.py", line 34, in reraise
raise value
File "D:\Python38\lib\site-packages\zmq\backend\__init__.py", line 27, in <module>
_ns = select_backend(first)
File "D:\Python38\lib\site-packages\zmq\backend\select.py", line 28, in select_backend
mod = __import__(name, fromlist=public_api)
File "D:\Python38\lib\site-packages\zmq\backend\cython\__init__.py", line 6, in <module>
from . import (constants, error, message, context,
ImportError: cannot import name 'constants' from partially initialized module 'zmq.backend.cython' (most likely due to a circular import) (D:\Python38\lib\site-packages\zmq\backend\cython\__init__.py)
I am also having this problem, on two systems. Fresh install of Python 3.9.1, pzmq 21.0.2. Not using Anaconda.
Installed as a dependency of jupyter, using pip.
> jupyter notebook
Traceback (most recent call last):
File "c:\users\me\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\me\appdata\local\programs\python\python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\me\AppData\Local\Programs\Python\Python39\Scripts\jupyter-notebook.EXE\__main__.py", line 4, in <module>
File "c:\users\me\appdata\local\programs\python\python39\lib\site-packages\notebook\notebookapp.py", line 49, in <module>
from zmq.eventloop import ioloop
File "c:\users\me\appdata\local\programs\python\python39\lib\site-packages\zmq\__init__.py", line 62, in <module>
from zmq import backend
File "c:\users\me\appdata\local\programs\python\python39\lib\site-packages\zmq\backend\__init__.py", line 32, in <module>
raise original_error from None
File "c:\users\me\appdata\local\programs\python\python39\lib\site-packages\zmq\backend\__init__.py", line 27, in <module>
_ns = select_backend(first)
File "c:\users\me\appdata\local\programs\python\python39\lib\site-packages\zmq\backend\select.py", line 32, in select_backend
mod = import_module(name)
File "c:\users\me\appdata\local\programs\python\python39\lib\importlib\__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "c:\users\me\appdata\local\programs\python\python39\lib\site-packages\zmq\backend\cython\__init__.py", line 6, in <module>
from . import (
ImportError: cannot import name 'constants' from partially initialized module 'zmq.backend.cython' (most likely due to a circular import) (c:\users\me\appdata\local\programs\python\python39\lib\site-packages\zmq\backend\cython\__init__.py)
Many other references to this issue on other repos, on Stack Overflow, etc. suggest uninstalling and reinstalling pyzmq. This doesn't resolve the issue for me. Thanks.
This is something to do with the Cython backend - I guess most people end up using the CFFI backend automatically and never hear about this failure.
If the Cython modules are not for the same version of Python that you are running it with, they'll fail to import. Because it happens as part of a complex, nested chain of imports, it is reported as a circular import rather than a straight failure.
In my case, the 3.9 binaries had installed into 3.8. I'm pretty sure I know why that happened, and it doesn't apply generally, so I'm not sure why others have run into it. Certainly using the wrong pip version could cause it, since pip assumes that it can just inspect its own runtime to determine which package to install. This would also explain why reinstalling fixes it for some people - they probably only got the version wrong once (or they may have hit the problem with running pip in a Visual Studio Command Prompt, which will make it install only 32-bit packages).
Thanks for looking into it @zooba!
I guess most people end up using the CFFI backend automatically and never hear about this failure.
That's very unlikely, as most folks don't have the compilers necessary to build the CFFI backend, so the fallback will fail more often than not.
The current wheels (22.0.3) import and run without issue in all testing I've managed to do. A Windows Server 2012 or 2019 machine with nothing but 32b Python 3.7. and 64b Python 3.9.5 install and run the current pyzmq wheels without any issues.
Is the issue only with Python in debug mode? I have little to no experience with how debug-Python behaves on Windows, so couldn't begin to guess what pyzmq should do differently, if anything.
If anyone can come up with a reproducible case where pyzmq does the wrong thing, I would love that. I've never managed create an environment where these wheels don't work, and don't have the Windows DLL expertise to understand what's unique about the environments where the import fails.
The issue was because the installed binaries didn't match the CPython version. I know a number of ways to make that happen, all of which are user error or bugs, and none of which are pyzmq's to resolve.
Right now I think the import failure is actually occurring during a circular import that would otherwise sort itself out. The only thing pyzmq could do is to simplify its initial import sequence, which could turn the error message from "circular import" into accurately specifying the module that is missing.
Yeah, perfect. I'm not sure when the circular import started happening, because it didn't not long ago.
What's weird is zmq.backend.cython.constants has no imports, it's purely a list of assignments (maybe Cython injects some, I'm not sure). And an __init__ doing from .submod import something is super normal, no?
Maybe something changed in error reporting, such that it's suggesting a circular import where there isn't one, instead of letting the underlying error raise?
At a guess, it might be that one of the submodule imports is trying to import one of the other submodules which requires re-importing the parent module, which is going to cause the circular import.
But it's an innocuous circle, because when everything works, it works. The problem is just that when an import fails, it is misreported as a circular import because we're inside one. I'm not sure whether we can fix CPython to tell the difference or not, because there's likely no way to generically tell whether the failure is due to the cycle or not.
And it may just be enough to know that if pyzmq is encountering the error, it might not be an actual circular dependency. That would have helped everyone in this thread, I think.
Thanks. I wonder what changed, because one of the wonderful things about moving from Python 2 to Python 3 is that we stopped seeing 'cannot import name constants', and started seeing the actual DLL errors which were debuggable. That was a huge improvement for folks who had some installation issues. But now we are back to opaque "it doesn't work, I won't tell you why," with the interpreter hiding the real error again.
Thanks for the hint about local cycles, maybe there's something I changed here that I can resolve.
Same issue here. I get that import error in a fresh Windows docker container base on an image built by the following Dockerfile:
FROM mcr.microsoft.com/windows:20H2
SHELL ["cmd", "/S", "/C"]
# Donwload version 16.9.12 from Microsoft
RUN curl -SL --output vs_buildtools.exe https://download.visualstudio.microsoft.com/download/pr/dcb0a070-a0c3-4fda-a07e-b00b4f777924/0f7200f381b269441a113ba7bf310c4dd291ffcc9e62d0a934dc9343add96a98/vs_BuildTools.exe
# Had some pretty weird behaviour if the folder below is missing (which it obviously is ... duh)
# Creating the folder beforehand
RUN mkdir "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
RUN start /w vs_buildtools.exe --quiet --wait --norestart --nocache`
--add Microsoft.VisualStudio.Workload.VCTools;includeRecommended`
--add Microsoft.VisualStudio.Component.VC.Llvm.Clang`
--add Microsoft.VisualStudio.Workload.ManagedDesktopBuildTools`
&& del /q vs_buildtools.exe
# Installing chocolatey AFTER the vs buildtools. This was also due to some issues with the installer, if it is run before the vs_buildtools installer
RUN powershell Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
RUN choco install -y cmake --version=3.16.5 --installargs 'ADD_CMAKE_TO_PATH=System' && `
choco install -y pwsh
ENTRYPOINT ["C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\Tools\\VsDevCmd.bat", "&&", "powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]
RUN choco install -y git --version=2.32.0.2 && \
choco install -y swig --version=4.0.2.04082020 && \
choco install -y doxygen.install --version=1.9.0 && \
choco install -y graphviz --version=2.49.1 && \
choco install -y wget --version=1.21.2 && \
choco install -y cmake --version=3.21.4 && \
choco install -y ninja --version=1.10.2 && \
choco install -y openssl --version=1.1.1.1100
RUN certutil -generateSSTFromWU roots.sst && certutil -addstore -f root roots.sst && del roots.sst
# install Python 3.8.10 (64bit)
RUN wget https://www.python.org/ftp/python/3.8.10/python-3.8.10-amd64.exe && \
echo "Install Python 3.8.10 (64bit)..." && \
.\python-3.8.10-amd64.exe /quiet InstallAllUsers=1 PrependPath=1 && \
"%ProgramFiles%\\Python38\\python.exe" -m pip install -Iv numpy==1.21.3 && \
echo "Remove Python 3.8.10 (64bit) installer..." && \
Del python-3.8.10-amd64.exe
RUN powershell "Get-Service -Name ssh-agent | Set-Service -StartupType Automatic"
RUN powershell git config --global core.sshCommand \"$((Get-Command ssh).Source.replace('\','\\')) -o StrictHostKeyChecking=no\"
CMD [ "pwsh.exe" ]
I discovered the error trying to run the following command sequence:
pip install nbconvert
jupyter nbconvert --help
The following command sequence triggers the same error however:
pip install pyzmq
python -c "import zmq"
Uhhh... me too: I have a MacPorts version of Jupyter installed that worked. I then installed a package using pip (big mistake right there, I guess), ended up with a mess, that results in jupyter notebook throwing the following error sequence ending with the ImportError. I have tried forced uninstall and reinstall of zmq, py-zmq, and py39-zmq.
dust-1742 67 % jupyter notebook
Traceback (most recent call last):
File "/Users/mordecai/Library/Python/3.8/bin/jupyter-notebook", line 5, in
I've built pyzmq on armv7l but I also see that error. Could it be that pyzmq cython/c code isn't compatible with that architecture?
I've built pyzmq on armv7l but I also see that error. Could it be that pyzmq cython/c code isn't compatible with that architecture?
It's not an architecture compatibility issue, but an installation conflict of some sort. This was all working fine for years until I used pip rashly.
I encounterd a very simmillar issue trying to use pyzmq in a Splunk app:
ImportError: cannot import name '_device' from 'zmq.backend.cython' (/Applications/Splunk/lib/python3.7/site-packages/zmq/backend/cython/__init__.py
The error occures independent of the installation directory of zmq. So installing it in the lib-folder of the app results in the same issue.
Versions:
- Splunk's python interepreter is of version 3.7.11
- zmq is 23.3.0
I had the same error when trying to run mu-editor. For me, it helped to upgrade my pip3 from v20 to the latest v23 and then to reinstall pyzmq
pip3 uninstall pyzmq
pip3 install --upgrade pip
~/.local/bin/pip install --no-cache-dir pyzmq
