flower icon indicating copy to clipboard operation
flower copied to clipboard
verified publisher icon mher

ImportError - Future exception during startup

Open snio89 opened this issue 4 years ago • 0 comments

Describe the bug The system stuck with the following repeated error just after starting the flower. I'm new to flower, and I first tried Celery 5.0.5 and Flower master version. After encountering this error, I changed to Celery 4.4.7 and Flower 0.9.7 but the same error occurred.

To Reproduce

  1. Run Ubuntu 20.04 Docker Image .
  2. Run RabbitMQ Docker Image (with the management plugin) as a Celery broker.
  3. Run Redis Docker Image as a Celery result backend.
  4. Configure Celery settings within the Django project configuration file.
  5. Start Celery worker as daemon: celery -A proj worker --detach
  6. Start Celery beat as daemon: celery -A proj beat --detach
  7. Start Flower with either celery -A proj flower --port=5555 --broker_api=http://rabbitmquser:rabbitmqpw@rabbitmqhost:15672/api/ or flower -A proj --port=5555 --broker_api=http://rabbitmquser:rabbitmqpw@rabbitmqhost:15672/api/

Expected behavior Flower has to be started with no error.

Screenshots

[I 210327 23:19:12 command:135] Visit me at http://127.0.0.1:5555
[I 210327 23:19:12 command:142] Broker: amqp://rabbitmquser:**@rabbitmqhost:5672/rabbitmqvhost
[I 210327 23:19:12 command:143] Registered tasks: 
    ['celery.accumulate',
     'celery.backend_cleanup',
     'celery.chain',
     'celery.chord',
     'celery.chord_unlock',
     'celery.chunks',
     'celery.group',
     'celery.map',
     'celery.starmap']
[E 210327 23:19:12 base_events:1707] Future exception was never retrieved
    future: <Future finished exception=ImportError('/usr/local/lib/python3.8/dist-packages/_librabbitmq.cpython-38-x86_64-linux-gnu.so: undefined symbol: basic_properties_to_PyDict')>
    Traceback (most recent call last):
      File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/usr/local/lib/python3.8/dist-packages/flower/inspector.py", line 38, in _inspect
        result = getattr(inspect, method)()
      File "/usr/local/lib/python3.8/dist-packages/celery/app/control.py", line 143, in active_queues
        return self._request('active_queues')
      File "/usr/local/lib/python3.8/dist-packages/celery/app/control.py", line 98, in _request
        return self._prepare(self.app.control.broadcast(
      File "/usr/local/lib/python3.8/dist-packages/celery/app/control.py", line 467, in broadcast
        with self.app.connection_or_acquire(connection) as conn:
      File "/usr/local/lib/python3.8/dist-packages/celery/utils/objects.py", line 87, in __enter__
        context = self._context = self.fallback(
      File "/usr/local/lib/python3.8/dist-packages/celery/app/base.py", line 845, in _acquire_connection
        return self.pool.acquire(block=True)
      File "/usr/local/lib/python3.8/dist-packages/kombu/resource.py", line 83, in acquire
        R = self.prepare(R)
      File "/usr/local/lib/python3.8/dist-packages/kombu/connection.py", line 1010, in prepare
        resource = resource()
      File "/usr/local/lib/python3.8/dist-packages/kombu/utils/functional.py", line 210, in __call__
        return self.evaluate()
      File "/usr/local/lib/python3.8/dist-packages/kombu/utils/functional.py", line 213, in evaluate
        return self._fun(*self._args, **self._kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kombu/connection.py", line 982, in new
        return self.connection.clone()
      File "/usr/local/lib/python3.8/dist-packages/kombu/connection.py", line 626, in clone
        return self.__class__(**dict(self._info(resolve=False), **kwargs))
      File "/usr/local/lib/python3.8/dist-packages/kombu/connection.py", line 636, in _info
        D = self.transport.default_connection_params
      File "/usr/local/lib/python3.8/dist-packages/kombu/connection.py", line 906, in transport
        self._transport = self.create_transport()
      File "/usr/local/lib/python3.8/dist-packages/kombu/connection.py", line 615, in create_transport
        return self.get_transport_cls()(client=self)
      File "/usr/local/lib/python3.8/dist-packages/kombu/connection.py", line 621, in get_transport_cls
        transport_cls = get_transport_cls(transport_cls)
      File "/usr/local/lib/python3.8/dist-packages/kombu/transport/__init__.py", line 87, in get_transport_cls
        _transport_cache[transport] = resolve_transport(transport)
      File "/usr/local/lib/python3.8/dist-packages/kombu/transport/__init__.py", line 72, in resolve_transport
        return symbol_by_name(transport)
      File "/usr/local/lib/python3.8/dist-packages/kombu/utils/imports.py", line 57, in symbol_by_name
        module = imp(module_name, package=package, **kwargs)
      File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
      File "<frozen importlib._bootstrap>", line 991, in _find_and_load
      File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 783, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/usr/local/lib/python3.8/dist-packages/kombu/transport/librabbitmq.py", line 11, in <module>
        import librabbitmq as amqp
      File "/usr/local/lib/python3.8/dist-packages/librabbitmq/__init__.py", line 7, in <module>
        import _librabbitmq
    ImportError: /usr/local/lib/python3.8/dist-packages/_librabbitmq.cpython-38-x86_64-linux-gnu.so: undefined symbol: basic_properties_to_PyDict
[E 210327 23:19:12 base_events:1707] Future exception was never retrieved
    future: <Future finished exception=ImportError('/usr/local/lib/python3.8/dist-packages/_librabbitmq.cpython-38-x86_64-linux-gnu.so: undefined symbol: basic_properties_to_PyDict')>
    Traceback (most recent call last):
      File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/usr/local/lib/python3.8/dist-packages/flower/inspector.py", line 38, in _inspect
        result = getattr(inspect, method)()
      File "/usr/local/lib/python3.8/dist-packages/celery/app/control.py", line 131, in revoked
        return self._request('revoked')
...

System information

flower   -> flower:0.9.7 tornado:6.1 humanize:3.3.0
software -> celery:4.4.7 (cliffs) kombu:4.6.11 py:3.8.5
            billiard:3.6.3.0 py-amqp:2.6.1
platform -> system:Linux arch:64bit, ELF
            kernel version:5.4.72-microsoft-standard-WSL2 imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

pip install requirements:

Django == 3.1.7
celery[librabbitmq,redis] == 4.4.7
django-celery-beat == 2.2.0
flower

Accessing http://rabbitmqhost:15672/ is possible.

Note 1 When using flower -A proj --port=5555 --broker_api=http://rabbitmquser:rabbitmqpw@rabbitmqhost:15672/api/ for Celery 5.0.5 and Flower master version, Flower simply does not work and tells the option port does not exist.

root@***:~/***# flower -A proj --port=5555 --broker_api=http://rabbitmquser:rabbitmqpw@rabbitmqhost:15672/api/
Usage: flower [OPTIONS] COMMAND [ARGS]...
Try 'flower --help' for help.

Error: no such option: --port

Using celery -A proj flower --port=5555 --broker_api=http://rabbitmquser:rabbitmqpw@rabbitmqhost:15672/api/ results in the same importError as above.

Note 2 Explicit cli option --broker=amqp:///..., --broker=pyamqp:///..., --broker=librabbitmq:///... did not help.

snio89 avatar Mar 27 '21 14:03 snio89