celery-director icon indicating copy to clipboard operation
celery-director copied to clipboard

Run the same task in parallel to x times?

Open abusquets opened this issue 3 years ago • 1 comments
trafficstars

Reading the documentation I know that celery-director can run tasks in parallel https://ovh.github.io/celery-director/guides/build-workflows/#launch-tasks-in-parallel But I have a question related to this feature. What is the definition to run the same task in parallel to x times?

Task A: retrieve list of news Task B: retrieve detail

Thanks

abusquets avatar Mar 19 '22 07:03 abusquets

Hi @abusquets ,

Unfortunately this feature is not possible in Celery Director as we don't allow dynamic workflows. The original objective of this project was to provide an easy way to build simple workflows, so dynamic workflows are not supported.

I keep your issue opened for future discussions. For instance we think to provide helpers used to create tasks in the code itself, it can be a solution for your usecase. But I prefer to be honest, it's not a priority for now and I'm not sure when we'll be able to provide this feature :)

ncrocfer avatar Mar 22 '22 11:03 ncrocfer