bullmq
bullmq copied to clipboard
Store the queues defaultJobOptions independently from the jobs to reduce memory consumption
I noticed that the defaultJobOptions are added to each job in redis,
I thought they would be stored independently from the jobs.
I think it would be nice to store the defaultJobOptions in the queue and not add them to each job in redis,
that would reduce the memory consumption e.g. when using custom job retain options for
completed and failed jobs.
Okay, I missed that we can at least do this for the retain options by passing them to the worker options as opposed to the queue options although that difference has not been documented.
https://docs.bullmq.io/guide/queues/auto-removal-of-jobs https://docs.bullmq.io/guide/workers/auto-removal-of-jobs
Yes, we are trying to move some settings to the workers to reduce memory usage.