agenda icon indicating copy to clipboard operation
agenda copied to clipboard

Jobs don't run when mongo has 1 job many times

Open CY-Christin opened this issue 2 years ago • 4 comments

Node: v14.9.0 agenda: ^2.0.2 what happened: when i add one job into mongo many times,(the jobs have the same name), agenda does not work . But when i add a job with another name, the new one will run normally. So i restart my server , 5 jobs run , the number is same as [defaultConcurrency]. However , old jobs are updating their [lockedAt] in the mongo, but they don't run. I wanner know why they don't run, i read the document , just find 'Also worth noting is that if the job queue is shutdown, any jobs stored in memory that haven't run will still be locked, meaning that you may have to wait for the lock to expire. By default it is '5 seconds', but i can't find anything about 'shutdown'. Is there anything happened about my job queue? Where can i see the problem? Thank you!

CY-Christin avatar May 11 '22 12:05 CY-Christin

Can you share the code that you used to schedule the job?

harisvsulaiman avatar May 12 '22 06:05 harisvsulaiman

schedule: async reconnectLiveStream({ time, userId }){ await agenda.schedule(new Date(time), 'relive jobs', { userId }); } define: agenda.define('relive jobs', { lockLiftTime: 1000 }, async (job, done) => { const data = job.attrs.data; console.log('relive job', data); await somefunction(data.userId) .then(done) .catch(err => { done(err); }); });

mongodb: testdb:PRIMARY> db.agendaJobs.find({}).sort({"nextRunAt":-1}).limit(5) { "_id" : ObjectId("627cb0989febe3001a1b7d79"), "name" : "relive job", "data" : { "userId" : "111111111" }, "type" : "normal", "priority" : 0, "nextRunAt" : ISODate("2022-05-12T08:00:40Z"), "lastModifiedBy" : null } { "_id" : ObjectId("627cbb009febe3001a1b7d8d"), "name" : "relive job", "data" : { "userId" : "111111111" }, "type" : "normal", "priority" : 0, "nextRunAt" : ISODate("2022-05-12T07:46:04Z"), "lastModifiedBy" : null, "lockedAt" : ISODate("2022-05-12T07:46:01.687Z") } { "_id" : ObjectId("627cbaea9febe3001a1b7d89"), "name" : "relive job", "data" : { "userId" : "111111111" }, "type" : "normal", "priority" : 0, "nextRunAt" : ISODate("2022-05-12T07:45:41Z"), "lastModifiedBy" : null, "lockedAt" : ISODate("2022-05-12T07:45:36.678Z") } { "_id" : ObjectId("627cb9899febe3001a1b7d85"), "name" : "relive job", "data" : { "userId" : "111111111" }, "type" : "normal", "priority" : 0, "nextRunAt" : ISODate("2022-05-12T07:39:49Z"), "lastModifiedBy" : null, "lockedAt" : ISODate("2022-05-12T07:49:46.758Z") } { "_id" : ObjectId("627cb9509febe3001a1b7d81"), "name" : "relive job", "data" : { "userId" : "111111111" }, "type" : "normal", "priority" : 0, "nextRunAt" : ISODate("2022-05-12T07:38:52Z"), "lastModifiedBy" : null, "lockedAt" : ISODate("2022-05-12T07:48:51.739Z") }

CY-Christin avatar May 12 '22 08:05 CY-Christin

Can you share the code that you used to schedule the job?

MongoDB just update lockedAt but doesn't work

CY-Christin avatar May 12 '22 08:05 CY-Christin

having the same issue i have 600,000 jobs, after running 70k i get to a situation where it only marked locked but do not process anymore defaultLockLifetime: 4000, lockLimit: 5 maxConcurrency: 30

eran-barkai avatar Dec 27 '22 05:12 eran-barkai