Repeated job using "every" starts at the wrong time

[ajwootto]

Description

Hi, I'm adding a job to Bull like this:

bull.add(data, {
     delay: 600000,
     repeat: {
         every: 600000,
         jobId: "some_string"
     }
}

Basically the job should happen for the first time 10 minutes from the time this is called, and then repeat every 10 minutes thereafter.

If I call this at, say, 4:03 PM, the job is first processed at 4:10 PM and repeats every 10 minutes from that point on.

It seems that the "every" parameter is just ignoring the relative delay and just having the job repeat on the 10 minute marks of the hour. Is this expected behaviour or should it actually be happening at 4:13 PM, 4:23 PM and so on?

[ajwootto]

On looking at the code, it seems that the scheduling for "every" parameter rounds the current date to the nearest interval of length "every": https://github.com/OptimalBits/bull/blob/develop/lib/repeatable.js#L170

As far as I can understand, this would mean that a repeating interval of one day would result in the job being scheduled to repeat at the beginning of each day, regardless of when the job is actually scheduled. Is this the desired behaviour?

If it was changed to something like

if (opts.every) {
      return Math.floor(millis / 60000) * 60000 + opts.every;
}

then the job's schedule would be rounded to the nearest minute and occur for the first time after that nearest minute plus the interval.

[manast]

It is actually the desired behaviour, and as far as I remember this behaviour is necessary to guarantee atomicity and the hazard that a given repeatable job could add itself more than 1 time per iteration.

[manast]

not sure if this applies to every actually, maybe only to standard cron based jobs :/

[ajwootto]

From what I can tell, the job basically keeps rescheduling itself each time it is processed, using the result of that computation to determine the new time. Wouldn't the atomicity be independent of which time is actually chosen next?

[ajwootto]

Just wanted to bump this to make sure it's actually the expected behaviour since it seems to me that this wouldn't be desired in the majority of cases.

[HugoPoi]

This thing is very confusing because the documentation is obviously incomplete. As in the documentation

Bull is smart enough not to add the same repeatable job if the repeat options are the same.

Repeatable jobs are not duplicate at the condition they are added in the same every time window.

With documentation example if you run 2 queue.add() in less than 10 seconds only 1 job will be inserted but if you run 2 queue.add() spaced out of more than 10 seconds then 2 jobs will be inserted. When a processor is available all the scheduled jobs will be executed in the good order and only one will reschedule. ( Edited )

[JanusSpark]

Just wanted to bump this to make sure it's actually the expected behaviour since it seems to me that this wouldn't be desired in the majority of cases.

every and cron options both first run are scheduled "on the hour" like repeated-job-details says. how do you solve this @ajwootto when the repeat job become more and more. It seems like a DDOS attack my job server...

[ajwootto]

@JanusSpark I ended up just creating a fork of the library that includes the patch I wrote above.