open-semantic-search icon indicating copy to clipboard operation
open-semantic-search copied to clipboard

Strange behaviour of Celery-Flower

Open mmoossen opened this issue 3 years ago • 2 comments

Hi, i realized a very strange behaviour of Celery-Flower.

  • I got 30.000+ files to index.
  • I clean the index with:
rabbitmqctl -n rabbit@search purge_queue open_semantic_etl_tasks
opensemanticsearch-delete --empty
  • i navigate to /search-apps/files/ and press "index now".
  • i can see files get indexed on the search page
  • i can also see the indexed files in the logs with
systemctl status opensemanticetl
  • and i also see the queued files with
rabbitmqctl -n rabbit@search list_queues

but MOST of the time, the celery-flower queue stays empty. image

And really MOST of the time, but not all the time. SOMETIMES celery-flower shows the process, but MOST of the time NOT. But always all files get indexed!

The same behaviour can also be observed with incremental indexing.

  1. have you had any behaviour like this?
  2. do you know what could be wrong?

many thanks Michael

mmoossen avatar Nov 02 '21 09:11 mmoossen

Which flavour of opensemanticsearch do you use? Celery flower is missing at all on my installation of open-semantic-search_21.12.26 on Debian 11.

samoal avatar Feb 14 '22 20:02 samoal

We use the last version available at that time: open-semantic-desktop-search_21.01.03.ova

And it is definitively installed:

# apt list --installed | grep celery
python3-celery/oldstable,now 4.2.1-3 all [installed,automatic]

And it is accessible from this button: image

moossenm avatar Feb 15 '22 05:02 moossenm