Queue doesn't process added job

[EizFeren]

Description

Queue doesn't process job after adding it - I've checked redis, and there was job data, but queue didn't process that job.

Minimal, Working Test code to reproduce the issue

import dotenv from 'dotenv';

dotenv.config();

import Queue from 'bull';

const queue = new Queue('test', {
  redis: {
    host: process.env.REDIS_HOST,
    port: process.env.REDIS_PORT,
    username: process.env.REDIS_USERNAME,
    password: process.env.REDIS_PASSWORD,
    database: process.env.REDIS_DATABASE,
    retryStrategy() {
      return 1000;
    },
    maxRetriesPerRequest: null,
    lazyConnect: true,
    enableReadyCheck: false,
  },
});

queue.process(async (job) => {
  console.log(job);
});

queue.add({
  test: 'test',
});

As a result - console.log(job) inside processor was never called.

Bull version

^4.2.0

Additional information

NodeJS version - v14.17.6 Redis version - v6.0.6

[yovanoc]

Same issue here

[bmnepali]

I have the same issue, this actually happens when my redis server restarts and the api is still up and running.

Using: @nestjs/bull 0.4.2 bull: 4.2.0

[manast]

@EizFeren please do not use the option lazyConnect: true. For some reason, it is not compatible with Bull.

[EizFeren]

Ok, ty. Please add this to docs

[AndyICT2U]

Hi @EizFeren Did the job still progress to completed status?

I am having similar behaviour but with the basic localhost redis settings of {redis: {port: 6379, host: '127.0.0.1'}} I can see jobs are being added to the queue (checking the getJobCounts() after performing the add) as an active state and sometimes these jobs go through the .process function without any problems and other times they don't. Strangely though all the jobs progress to 'completed' state so I am a little confused. I'm not able to identify an obvious pattern as to whether a job will progress or not. I have added a variety of debugging but can't identify what is happening to those odd jobs that don't seem to be processed but do get completed. I should add I'm on Bull v3.4.8 and a little constrained to update to latest versions currently unless this is a known bug.

Thanks for any clues as to whether this is an issue or if there are extra debug messages that I could consider to trap why this is happening.

Regards Andy

I should add I'm running two separate queues against that same redis server if that has any bearing. The other queue seems to be working without any issue.

[AndyICT2U]

Brief update - looks like it was down to AWS degrading the server that it was running on so probably a resource-constrained outcome although intrigued as to why it manifested like this.

[udaya-reddy]

Hi, This issue seem to still occur. I am using Bull 4.11.5 and I use the queue for delayed jobs. sometimes the queue.process function gets called for most of queued jobs, but other times it doesnt. The job however is marked as completed and not removed from the completedJobs list. The ones where queue.process is called the jobs are not in queue as these are auto removed once completed. I use default settings ( not providing any overrides ) on the queue. `

          const redisOptions: any = {
  	host: config.redis.url,
  	port: config.redis.port,
  	password: config.redis.password,
  }
  if (config.redis.port != 6379 && config.redis.port != '6379') {
  	redisOptions.tls = true
  	redisOptions.enableTLSForSentinelMode = false
  }
           queue = new Queue(`scheduler`, {
  	redis: redisOptions,
  })

`

				queue.process(config.queueConcurrency || 20, async (job, done) => {
					logger.debug(`Process: Event ${job.data.name} triggered: ${job.data.id}. Proceeding to confirm.`)
      done & done()
})

1. Is this a bug where the status is completed but queue.process function wasnt called? at the least they should be in waiting state.
2. or, is there something else that is being missed here.

[FlavioCoutinhoGO]

Same issue here bull 4.12.2 Node v20.11.0

[manast]

I am closing this issue as it is quite old. If you want us to investigate further please create a new issue with some code that reproduces it, otherwise we do not have any actions to take... also, if you can, try to use BullMQ instead.