bullmq icon indicating copy to clipboard operation
bullmq copied to clipboard

Store the queues defaultJobOptions independently from the jobs to reduce memory consumption

Open lukas-becker0 opened this issue 1 year ago • 2 comments

I noticed that the defaultJobOptions are added to each job in redis, I thought they would be stored independently from the jobs.

I think it would be nice to store the defaultJobOptions in the queue and not add them to each job in redis, that would reduce the memory consumption e.g. when using custom job retain options for completed and failed jobs.

lukas-becker0 avatar Jun 27 '24 19:06 lukas-becker0

Okay, I missed that we can at least do this for the retain options by passing them to the worker options as opposed to the queue options although that difference has not been documented.

https://docs.bullmq.io/guide/queues/auto-removal-of-jobs https://docs.bullmq.io/guide/workers/auto-removal-of-jobs

lukas-becker0 avatar Jul 01 '24 11:07 lukas-becker0

Yes, we are trying to move some settings to the workers to reduce memory usage.

manast avatar Jul 02 '24 20:07 manast