ceps
ceps copied to clipboard
celery
Celery Enhancement Proposals
Points for discussion: ### CI - [ ] Code Linters - [ ] Test Runner & Required Plugins - [ ] Definition of Unit Tests - [ ] Definition of...
Several requests for help or debugging are opened as GitHub issues. The recommended communication methods are listed [here](https://docs.celeryproject.org/en/stable/getting-started/resources.html#getting-help), and possibly some of them are outdated, for example the `Wiki` link....
Celery's website looks outdated and probably contains a lot of irrelevant/outdated information. We should redesign it and ensure it fits our demands and promotes our projects (and more importantly the...
I've recently been looking at using celery for chained tasks or tasks with callbacks, where one or more of the tasks in the chain interacts with an external service that...
Points for discussion: - [ ] Release Issue - [ ] Release Milestone - [ ] Release Instructions Documentation - [ ] Release Schedule - [ ] Notification Channels -...
Default to broker_pool_limit = 1. There is no point in having many connections to a RabbitMQ server from a single process. Instead each worker/thread should have their own channel.
Default event_queue_expires = 60 The default of 24h is not good because sometimes when ev-queues are not consumed from they wouldn't be deleted until 24h later, and that could case...
The RPC backend should use the very fast direct-reply feature in RabbitMQ: https://www.rabbitmq.com/direct-reply-to.html instead of creating a new queue for each result. Especially for clustered server creating a deleting queues...
Default to worker_prefetch_multiplier = 1 Most celery workers doesn't have a throughput that justifies a large prefetch amount, and second, you use AMQP global prefetch instead of consumer level prefetch,...
With many workers, and each sending a heartbeat message every 2s the accumulated throughput can be tens of thousands of msgs/s just for just heartbeats, which are completely redundant, as...