celery-batches
celery-batches copied to clipboard
flower support
Hello! How can I use it with flower? "create_task" works fine with flower, but I can't see "count_click" tasks on flower. What could I do wrong? It feels like I'm using wrong namespaces or starting arguments.
from .celery import celery_app
import time
from celery_batches import Batches, SimpleRequest
from collections import Counter
@celery_app.task(name="create_task")
def create_task(task_type):
time.sleep(int(task_type) * 2)
return True
@celery_app.task(base=Batches, flush_every=5, flush_interval=2)
def count_click(requests: list[SimpleRequest]):
"""Count the number of times each URL is requested."""
count = Counter(request.kwargs["click"] for request in requests)
for url, count in count.items():
print(f">>> Clicks: {url} -> {count}")
for request in requests:
celery_app.backend.mark_as_done(request.id, 'hi', request=request)
A working example would be appreciated.
celery-batches unfortunately doesn't provide all the signals that normal Celery tasks do (see #33 for an example). I suspect this is why you're not seeing it appear properly in flower. Does flower have a list of signals it depends on?
Thinking a bit more about this, I wonder what the appropriate events to send are. For example if we call a task 30 times with a batch size of 10; should we report that each task request started, finished, etc. or should we only report each batch? I suspect the latter makes more sense.