pyzmq
pyzmq copied to clipboard
Published
20 hours ago •
zeromq
zeromq
ImportError after compiling with build_ext --inplace: "...zmq.backend.cython.socket.Socket has the wrong size..."
Hi, I ran into this whilst trying to fix issue #844. I checked out master, ran python setup.py fetch_libzmq, and compiled from source using python setup.py build_ext --inplace. This seemed to work (though note the warning below in the compilation output: "Detected ZMQ version: 4.2.3, but pyzmq targets ZMQ 4.3.1.") The bundled version is 4.3.1 based on bundled/zeromq/include/zmq.h, but 4.2.3 is the libzmq version I have installed on my system in /usr/local. So that's the first suspicious thing.
Anyway, it fails when running python setup.py test with this error:
$ python setup.py test
/usr/local/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)
running test
Traceback (most recent call last):
File "setup.py", line 823, in run
import zmq
File "/home/aaron/work/pyzmq/zmq/__init__.py", line 47, in <module>
from zmq import backend
File "/home/aaron/work/pyzmq/zmq/backend/__init__.py", line 40, in <module>
reraise(*exc_info)
File "/home/aaron/work/pyzmq/zmq/utils/sixcerpt.py", line 34, in reraise
raise value
File "/home/aaron/work/pyzmq/zmq/backend/__init__.py", line 27, in <module>
_ns = select_backend(first)
File "/home/aaron/work/pyzmq/zmq/backend/select.py", line 35, in select_backend
reraise(ImportError, ImportError("Importing %s failed with %s" % (name, e)), exc_info[2])
File "/home/aaron/work/pyzmq/zmq/utils/sixcerpt.py", line 33, in reraise
raise value.with_traceback(tb)
File "/home/aaron/work/pyzmq/zmq/backend/select.py", line 28, in select_backend
mod = __import__(name, fromlist=public_api)
File "/home/aaron/work/pyzmq/zmq/backend/cython/__init__.py", line 6, in <module>
from . import (constants, error, message, context,
File "zmq/backend/cython/socket.pxd", line 33, in init zmq.backend.cython._device (zmq/backend/cython/_device.c:2624)
cdef class Socket:
ImportError: Importing zmq.backend.cython failed with zmq.backend.cython.socket.Socket has the wrong size, try recompiling. Expected 72, got 64
Fatal: Could not import zmq!
You must build pyzmq with 'python setup.py build_ext --inplace' for 'python setup.py test' to work.
If you did build pyzmq in-place, then this is a real error.
The same happens when simply trying to import zmq from a python prompt.
My suspicion is that somehow pyzmq is picking up my system's local libzmq instead of the bundled one, but I have not had much success trying to figure out whether this is the case or whether there is a way to force it to use the right libzmq. I'd appreciate any help anyone can give!
Here is my python version:
Python-3.5.6 (default, Jan 24 2019, 18:26:07)
[GCC 8.2.1 20181127]
Here's the output of python setup.py build_ext --inplace.
running build_ext
running configure
/usr/lib64/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)
Settings obtained from pkg-config: {'library_dirs': [], 'include_dirs': [], 'libraries': ['zmq']}
{'library_dirs': [], 'include_dirs': [], 'runtime_library_dirs': [], 'extra_link_args': [], 'libraries': ['zmq']}
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prot
otypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -c build/temp.linux-x86_64-3.5/scratch/check_sys_un.c -o buil
d/temp.linux-x86_64-3.5/scratch/check_sys_un.o
gcc -pthread build/temp.linux-x86_64-3.5/scratch/check_sys_un.o -o build/temp.linux-x86_64-3.5/scratch/check_sys_un
Configure: Autodetecting ZMQ settings...
Custom ZMQ dir:
************************************************
creating build/temp.linux-x86_64-3.5/scratch/tmp
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prot
otypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -c /tmp/timer_createei_gsrdc.c -o build/temp.linux-x86_64-3.5
/scratch/tmp/timer_createei_gsrdc.o
/tmp/timer_createei_gsrdc.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main (int argc, char **argv) {
^~~~
/tmp/timer_createei_gsrdc.c: In function ‘main’:
/tmp/timer_createei_gsrdc.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration
]
timer_create();
^~~~~~~~~~~~
gcc -pthread build/temp.linux-x86_64-3.5/scratch/tmp/timer_createei_gsrdc.o -o build/temp.linux-x86_64-3.5/scratch/a.out
/usr/bin/ld: build/temp.linux-x86_64-3.5/scratch/tmp/timer_createei_gsrdc.o: in function `main':
/tmp/timer_createei_gsrdc.c:2: undefined reference to `timer_create'
collect2: error: ld returned 1 exit status
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prot
otypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.
linux-x86_64-3.5/scratch/vers.c -o build/temp.linux-x86_64-3.5/scratch/vers.o
gcc -pthread build/temp.linux-x86_64-3.5/scratch/vers.o -lzmq -lrt -o build/temp.linux-x86_64-3.5/scratch/vers
ZMQ version detected: 4.2.3
Warning: Detected ZMQ version: 4.2.3, but pyzmq targets ZMQ 4.3.1.
Warning: libzmq features and fixes introduced after 4.2.3 will be unavailable.
************************************************
************************************************
skipping 'zmq/devices/monitoredqueue.c' Cython extension (up-to-date)
building 'zmq.devices.monitoredqueue' extension
creating build/temp.linux-x86_64-3.5/zmq
creating build/temp.linux-x86_64-3.5/zmq/devices
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prot
otypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devi
ces -I/usr/include/python3.5m -c zmq/devices/monitoredqueue.c -o build/temp.linux-x86_64-3.5/zmq/devices/monitoredqueue.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/devices/monitoredqu
eue.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/devices/monitoredqueue.cpython-35m-x86_64-linux-gnu
.so
skipping 'zmq/backend/cython/_device.c' Cython extension (up-to-date)
building 'zmq.backend.cython._device' extension
creating build/temp.linux-x86_64-3.5/zmq/backend
creating build/temp.linux-x86_64-3.5/zmq/backend/cython
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prot
otypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devi
ces -I/usr/include/python3.5m -c zmq/backend/cython/_device.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/_device.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/_dev
ice.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/_device.cpython-35m-x86_64-linux-gnu
.so
skipping 'zmq/backend/cython/_poll.c' Cython extension (up-to-date)
building 'zmq.backend.cython._poll' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prot
otypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devi
ces -I/usr/include/python3.5m -c zmq/backend/cython/_poll.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/_poll.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/_pol
l.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/_poll.cpython-35m-x86_64-linux-gnu.so
skipping 'zmq/backend/cython/_proxy_steerable.c' Cython extension (up-to-date)
building 'zmq.backend.cython._proxy_steerable' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devices -I/usr/include/python3.5m -c zmq/backend/cython/_proxy_steerable.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/_proxy_steerable.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/_proxy_steerable.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/_proxy_steerable.cpython-35m-x86_64-linux-gnu.so
skipping 'zmq/backend/cython/_version.c' Cython extension (up-to-date)
building 'zmq.backend.cython._version' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devices -I/usr/include/python3.5m -c zmq/backend/cython/_version.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/_version.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/_version.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/_version.cpython-35m-x86_64-linux-gnu.so
skipping 'zmq/backend/cython/constants.c' Cython extension (up-to-date)
building 'zmq.backend.cython.constants' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devices -I/usr/include/python3.5m -c zmq/backend/cython/constants.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/constants.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/constants.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/constants.cpython-35m-x86_64-linux-gnu.so
skipping 'zmq/backend/cython/context.c' Cython extension (up-to-date)
building 'zmq.backend.cython.context' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devices -I/usr/include/python3.5m -c zmq/backend/cython/context.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/context.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/context.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/context.cpython-35m-x86_64-linux-gnu.so
skipping 'zmq/backend/cython/error.c' Cython extension (up-to-date)
building 'zmq.backend.cython.error' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devices -I/usr/include/python3.5m -c zmq/backend/cython/error.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/error.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/error.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/error.cpython-35m-x86_64-linux-gnu.so
skipping 'zmq/backend/cython/message.c' Cython extension (up-to-date)
building 'zmq.backend.cython.message' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devices -I/usr/include/python3.5m -c zmq/backend/cython/message.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/message.o
zmq/backend/cython/message.c: In function ‘__pyx_pf_3zmq_7backend_6cython_7message_5Frame_16__str__’:
zmq/backend/cython/message.c:3898:33: warning: self-comparison always evaluates to true [-Wtautological-compare]
__pyx_t_3 = ((&PyString_Type) == (&PyUnicode_Type));
^~
zmq/backend/cython/message.c: In function ‘__pyx_pf_3zmq_7backend_6cython_7message_5Frame_18set’:
zmq/backend/cython/message.c:4329:18: warning: variable ‘__pyx_v_routing_id’ set but not used [-Wunused-but-set-variable]
pyzmq_uint32_t __pyx_v_routing_id;
^~~~~~~~~~~~~~~~~~
In function ‘__pyx_pf_3zmq_7backend_6cython_7message_5Frame_20get’,
inlined from ‘__pyx_pw_3zmq_7backend_6cython_7message_5Frame_21get’ at zmq/backend/cython/message.c:4558:13:
zmq/backend/cython/message.c:4789:55: warning: argument 1 null where non-null expected [-Wnonnull]
__pyx_t_4 = __Pyx_decode_c_string(__pyx_v_buf, 0, strlen(__pyx_v_buf), NULL, NULL, PyUnicode_DecodeUTF8); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 403, __pyx_L1_error)
^~~~~~~~~~~~~~~~~~~
In file included from /usr/include/python3.5m/Python.h:30,
from zmq/backend/cython/message.c:4:
zmq/backend/cython/message.c: In function ‘__pyx_pw_3zmq_7backend_6cython_7message_5Frame_21get’:
/usr/include/string.h:384:15: note: in a call to function ‘strlen’ declared here
extern size_t strlen (const char *__s)
^~~~~~
In function ‘__Pyx_decode_c_string.constprop’,
inlined from ‘__pyx_pf_3zmq_7backend_6cython_7message_5Frame_20get’ at zmq/backend/cython/message.c:4789:17,
inlined from ‘__pyx_pw_3zmq_7backend_6cython_7message_5Frame_21get’ at zmq/backend/cython/message.c:4558:13:
zmq/backend/cython/message.c:9764:23: warning: argument 1 null where non-null expected [-Wnonnull]
size_t slen = strlen(cstring);
^~~~~~~~~~~~~~~
In file included from /usr/include/python3.5m/Python.h:30,
from zmq/backend/cython/message.c:4:
zmq/backend/cython/message.c: In function ‘__pyx_pw_3zmq_7backend_6cython_7message_5Frame_21get’:
/usr/include/string.h:384:15: note: in a call to function ‘strlen’ declared here
extern size_t strlen (const char *__s)
^~~~~~
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/message.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/message.cpython-35m-x86_64-linux-gnu.so
skipping 'zmq/backend/cython/socket.c' Cython extension (up-to-date)
building 'zmq.backend.cython.socket' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devices -I/usr/include/python3.5m -c zmq/backend/cython/socket.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/socket.o
zmq/backend/cython/socket.c: In function ‘__pyx_pf_3zmq_7backend_6cython_6socket_6Socket_12bind’:
zmq/backend/cython/socket.c:6009:37: warning: self-comparison always evaluates to true [-Wtautological-compare]
__pyx_t_2 = ((&PyString_Type) == (&PyUnicode_Type));
^~
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/socket.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/socket.cpython-35m-x86_64-linux-gnu.so
skipping 'zmq/backend/cython/utils.c' Cython extension (up-to-date)
building 'zmq.backend.cython.utils' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fPIC -DHAVE_SYS_UN_H=1 -Izmq/utils -Izmq/backend/cython -Izmq/devices -I/usr/include/python3.5m -c zmq/backend/cython/utils.c -o build/temp.linux-x86_64-3.5/zmq/backend/cython/utils.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.5/zmq/backend/cython/utils.o -L/usr/lib -lzmq -lrt -lpython3.5m -o /home/aaron/work/pyzmq/zmq/backend/cython/utils.cpython-35m-x86_64-linux-gnu.so
