bullmq icon indicating copy to clipboard operation
bullmq copied to clipboard

[Bug]: upsertJobScheduler not upserting jobs at times

Open austinwoon opened this issue 9 months ago • 5 comments

Version

5.34.10

Platform

NodeJS

What happened?

upsertJobScheduler does not upsert jobs at times. At each redeploy, sometimes the job scheduler does not insert new jobs and no jobs get scheduled as a result. This happens undetermisitically for us

How to reproduce.

All we're doing is calling this at our index.ts fastify app file

    await this.queue.upsertJobScheduler(
      pollingWorkerName,
      {
        every: this.options.pollingFrequencyInMs ?? 2000,
      },
      {
        opts: {
          removeOnComplete: {
            count: 10000,
          },
        },
      }
    )

Relevant log output


Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

austinwoon avatar Feb 18 '25 04:02 austinwoon

There are several fixes in newer versions related to the job scheduler, I think that if you upgrade to the latest version you will not be able to reproduce this issue anymore, let me know how it goes.

manast avatar Feb 19 '25 21:02 manast

There are several fixes in newer versions related to the job scheduler, I think that if you upgrade to the latest version you will not be able to reproduce this issue anymore, let me know how it goes.

We just bumped the version to latest and still encountered same issue

Could it be because we have 4 instances trying to upsertJobScheduler at once?

austinwoon avatar Feb 20 '25 06:02 austinwoon

Facing the same issue on 5.43.1 where two instances are trying to upsertJobScheduler simultaneously.

Actions Taken We have upgraded to version 5.47.2 and will report back with results.

Reproduction Context This issue only occurs during our deployment pipeline when workers are restarted.

Will report back.

rohanbojja avatar Apr 07 '25 19:04 rohanbojja

Please report back as we think it is fixed in the latest releases.

manast avatar Apr 07 '25 19:04 manast

Fixed for us, bumping the version, and re-deploying created the jobs again. 👍🏼

rohanbojja avatar Apr 07 '25 19:04 rohanbojja

as looks like this issue is fixed, I'll proceed to close this issue. Pls if you encounter this issue again, feel free to reopen it or create a new one

roggervalf avatar May 18 '25 06:05 roggervalf