Pyzmq - from . import (constants, error, message, context, (Termux on Android)

[brubakerjeff]

Hi, I'm working on a program that imports Pyzmq. I upgraded to pyzmq-17.1.2 installed through pip. The base program is running through python 2.7, inside of termux and pyzmq was installed through pip. I'm receiving the following error. I isolated it to the following. Any help or pointers would be appreciated. Should the python3.7 work with python2.7 as well, or should I perhaps be using an older version?

pyzmq is installed using pip install pyzmq (also tried with adding --install-option="--zmq=bundled")

Is this configuration working a zmq dependency? Is there an easy way to install that with termux?

--- test .py

import zmq
print('do nothing')

$ python2 test.py 
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import zmq
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/__init__.py", line 47, in <module>
    from zmq import backend
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/__init__.py", line 40, in <module>
    reraise(*exc_info)
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/__init__.py", line 27, in <module>
    _ns = select_backend(first)
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/select.py", line 27, in select_backend
    mod = __import__(name, fromlist=public_api)
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/cython/__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: cannot import name constants

python3 appears to give more details

python3 test.py 
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import zmq
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/__init__.py", line 47, in <module>
    from zmq import backend
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/__init__.py", line 40, in <module>
    reraise(*exc_info)
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/utils/sixcerpt.py", line 34, in reraise
    raise value
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/__init__.py", line 27, in <module>
    _ns = select_backend(first)
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/select.py", line 27, in select_backend
    mod = __import__(name, fromlist=public_api)
  File "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/cython/__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: dlopen failed: cannot locate symbol "zmq_strerror" referenced by "/data/data/com.termux/files/usr/lib/python3.7/site-packages/zmq/backend/cython/error.cpython-37m.so"...

[minrk]

They key is probably that Python 2 is importing Python 3 packages. That's not going to work. Can you find why your Python 2 is importing from .../python3.7/site-packages? Can you uninstall pyzmq and include the full output of pip install --no-cache -v pyzmq? Do you have any *PATH environment variables set (env | grep PATH | sort) that might be relevant?

[brubakerjeff]

Thank you. Trying to understand this better. Does pip perform low level compilation tasks?

This makes me think it is a low level library compatibility concern.

testzmq.py #!/usr/bin/env python2.7 import zmq

running python2 testzmq.py produces ....

ImportError: dlopen failed: cannot locate symbol "zmq_strerror" referenced by "/data/data/com.termux/files/usr/lib/python2.7/site-packages/zmq/backend/cython/error.so"...

From pip install this is a bit suspicious but it completes anyway? ************************************************ Warning: Couldn't find an acceptable libzmq on the system. If you expected pyzmq to link against an installed libzmq, please check to make sure:

    * You have a C compiler installed
    * A development version of Python is installed (including headers)
    * A development version of ZMQ >= 3.2 is installed (including headers)
    * If ZMQ is not in a default location, supply the argument --zmq=<path>
    * If you did recently install ZMQ to a default location,
      try rebuilding the ld cache with `sudo ldconfig`
      or specify zmq's location with `--zmq=/usr/local`

Full output. https://gist.github.com/brubakerjeff/e34bfc4b0cefc772bb981e42f5e4a536

[minrk]

Ah, this is arm on android. That's probably relevant, and might have some consequences for building. You might have better luck building libzmq itself from source and then linking pyzmq to it instead.

[brubakerjeff]

Received below. I created an issue under zeromq. Looks like the call needs to be updated for this platform. I can experiment more tomorrow. They are pushing me to libzmq branch which is different than these instructions http://zeromq.org/build:android https://github.com/zeromq/zeromq3-x/issues/128

git clone https://github.com/zeromq/zeromq3-x.git
cd zeromq3-x/
./autogen.sh
./configure --enable-static --disable-shared --host=arm-linux-androideabi --prefix=$OUTPUT_DIR LDFLAGS="-L$OUTPUT_DIR/lib" CPPFLAGS="-fPIC -I$OUTPUT_DIR/include" LIBS="-lgcc"
make
make install
Making all in src
make[1]: Entering directory '/data/data/com.termux/files/home/zeromq3-x/src'
make  all-am
make[2]: Entering directory '/data/data/com.termux/files/home/zeromq3-x/src'
  CXX      libzmq_la-address.lo
  CXX      libzmq_la-clock.lo
  CXX      libzmq_la-ctx.lo
  CXX      libzmq_la-decoder.lo
  CXX      libzmq_la-devpoll.lo
  CXX      libzmq_la-dist.lo
  CXX      libzmq_la-encoder.lo
  CXX      libzmq_la-epoll.lo
  CXX      libzmq_la-err.lo
  CXX      libzmq_la-fq.lo
  CXX      libzmq_la-io_object.lo
  CXX      libzmq_la-io_thread.lo
  CXX      libzmq_la-ip.lo
  CXX      libzmq_la-ipc_address.lo
  CXX      libzmq_la-ipc_connecter.lo
  CXX      libzmq_la-ipc_listener.lo
ipc_listener.cpp:124:17: error: 'tempnam' is deprecated: tempnam is unsafe, use
      mkstemp or tmpfile instead [-Werror,-Wdeprecated-declarations]
        addr_ = tempnam(NULL, NULL);
                ^
/data/data/com.termux/files/usr/include/stdio.h:173:5: note: 'tempnam' has been
      explicitly marked deprecated here
    __warnattr("tempnam is unsafe, use mkstemp or tmpfile instead");
    ^
/data/data/com.termux/files/usr/include/sys/cdefs.h:160:40: note: expanded from
      macro '__warnattr'
#define __warnattr(msg) __attribute__((deprecated(msg)))
                                       ^
1 error generated.
make[2]: *** [Makefile:878: libzmq_la-ipc_listener.lo] Error 1
make[2]: Leaving directory '/data/data/com.termux/files/home/zeromq3-x/src'
make[1]: *** [Makefile:590: all] Error 2
make[1]: Leaving directory '/data/data/com.termux/files/home/zeromq3-x/src'
make: *** [Makefile:418: all-recursive] Error 1
$ make install
Making install in src
make[1]: Entering directory '/data/data/com.termux/files/home/zeromq3-x/src'
  CXX      libzmq_la-ipc_listener.lo
ipc_listener.cpp:124:17: error: 'tempnam' is deprecated: tempnam is unsafe, use
      mkstemp or tmpfile instead [-Werror,-Wdeprecated-declarations]
        addr_ = tempnam(NULL, NULL);
                ^
/data/data/com.termux/files/usr/include/stdio.h:173:5: note: 'tempnam' has been
      explicitly marked deprecated here
    __warnattr("tempnam is unsafe, use mkstemp or tmpfile instead");
    ^
/data/data/com.termux/files/usr/include/sys/cdefs.h:160:40: note: expanded from
      macro '__warnattr'
#define __warnattr(msg) __attribute__((deprecated(msg)))
                                       ^
1 error generated.
make[1]: *** [Makefile:878: libzmq_la-ipc_listener.lo] Error 1
make[1]: Leaving directory '/data/data/com.termux/files/home/zeromq3-x/src'
make: *** [Makefile:418: install-recursive] Error 1

[minrk]

It might be worth getting the latest (4.2.5) sources for libzmq as linked here. 3.x is pretty far back now.

[brubakerjeff]

worked towards 4.2.5 now have this https://github.com/zeromq/libzmq/issues/3307 I was able to install it.

[brubakerjeff]

Here are my commands now. Started from clean image git clone libzmq ./autogen.sh (change sh to bash inside of configure) ./configure --prefix=/data/data/com.termux/files/usr Makefile:1866: LIBS = -lrt -lpthread *-latomic * (added latomic, submitted PR) make N=1 make install git clone pyzmq cd pyzmq python2 setup.py configure --zmq=/data/data/com.termux/files/usr/lib python2 setup.py build_ext --inplace python2 setupegg.py develop (run cython on _device.pyx)

$ python2 Python 2.7.15 (default, Sep 23 2018, 14:55:02) [GCC 4.2.1 Compatible Android (4751641 based on r328903) Clang 7.0.2 (https://a on linux2 Type "help", "copyright", "credits" or "license" for more information.

import zmq zmq.zmq_version() '4.2.5'

Victory!