[Bug]: Queue's 'cleaned' event never fires.

[vidjuheffex]

Version

5.26.2

Platform

NodeJS

What happened?

Here: https://api.docs.bullmq.io/interfaces/v5.QueueEventsListener.html#cleaned

It says a cleaned event is fired when: jobs are cleaned (e.g., removed) from the queue, typically via a cleanup method.

I have removeOnFail and removeOnComplete set and I do see them get removed, but I can't attach an event to that, I'd assume, it'd be "cleaned", though I've tried "removed" as well, and neither of these ever fire.

I'm aware it happens lazily and all, but again, I see the jobs go away, an event for that just never fies.

How to reproduce.

function initQueueCleanup() { logger.info("Configuring amiRequestQueue cleanup"); amiRequestQueue.on("cleaned", async (_, id) => { logger.warn( Cleaning up job directory for job ID: ${id} from request queue on cleaned event ); const jobDirectory = await getJobDirectory(id);

try {
  // Attempt to remove the job directory and its contents recursively.
  await rm(jobDirectory, { recursive: true, force: true });
} catch (err: unknown) {
  // If the error indicates the directory doesn't exist, log a warning.
  if (err instanceof Error) {
    const errno = (err as NodeJS.ErrnoException).code;
    if (errno === "ENOENT") {
      logger.warn(`Job directory not found: ${jobDirectory}`);
    } else {
      logger.error(`Error removing job directory: ${jobDirectory}`, err);
    }
  } else {
    logger.error(`Error removing job directory: ${jobDirectory}`, err);
  }
}

});

amiRequestQueue.on("removed", async (job) => { logger.warn( Cleaning up job directory for job ID: ${job.id} from request queue on remove event ); const jobDirectory = job.data.jobDirectory;

try {
  // Attempt to remove the job directory and its contents recursively.
  await rm(jobDirectory, { recursive: true, force: true });
} catch (err: unknown) {
  // If the error indicates the directory doesn't exist, log a warning.
  if (err instanceof Error) {
    const errno = (err as NodeJS.ErrnoException).code;
    if (errno === "ENOENT") {
      logger.warn(`Job directory not found: ${jobDirectory}`);
    } else {
      logger.error(`Error removing job directory: ${jobDirectory}`, err);
    }
  } else {
    logger.error(`Error removing job directory: ${jobDirectory}`, err);
  }
}

}); }

void initQueueCleanup();

Relevant log output

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[manast]

Thanks for the issue. The "cleaned" event is only emitted when the queue is cleaned with the "clean" method, not when jobs are auto-removed due to removeOnComplete/Fail settings. Currently we do not emit any event when a job is removed using this settings. I wonder, what is the reason you need to be notified about a job that has been removed after finalising its processing?

[vidjuheffex]

Thanks for clarifying.

My jobs use the "steps" pattern outlined in the docs

So when a request comes in it enters the preparing step, in this step a job folder is created.

Once the job is removed I want to cleanup these temp directories, I was hoping to take care of this when a removed event is fired.

[manast]

I see. For jobs that complete, you could do the clean up as the last step, for failing jobs I think you should try to do the clean up by retrying the job automatically and ultimately by hand. Listening to a cleaned event would not give you any guarantees in the sense that 1) you could miss that event, and 2) the cleanup code inside the event could fail, leaving you without any means to perform the clean up later on.

[vidjuheffex]

That's the part I'm having trouble wrapping my head around re: cleaning on complete:

1. Job creates /tmp-folder-with-job-id/some-file.pdf
2. PDF creation finishes so job completes
3. Folder gets deleted
4. User can't get the file as soon as the job completes

I could make a finished job queue, copy the files to finished jobs, but I just kick the can down the road in terms of files that still need to be purged.

It really looked like removedOnComplete would give me an ideal setup of setting up a download window via maxAge and then when its done, cleaning itself. And I do feel it should emit a "removed" event, maybe passing a flag for when it was manual or automatic, but as a dev a remove-is-a-remove and my expectation was certainly that it would fire the event.

I hear you on "missing the event", so yeah ideally the jobs would self-cleanup and then a longer running job (24hrs) could manually purge the parent folder of any stragglers, as-is, the scheduled job has to run every 30minutes or so.

Thank you for the replies and insight!

[CaitW]

I would also love an event that fires when a job is auto-removed. I'm trying to debug an issue where jobs never seem to get removed despite setting options like removeOnComplete, removeOnFail, but it's difficult to do so. Right now I'm manually checking redis keys and comparing them over time, which is tedious.

[manast]

@CaitW if you are using "age" for removeOnComplete/removeOnFail, please notice that the jobs are being removed lazily, so this can lead to confusion as jobs that are older than age may still be in complete/fail status if no new job has been processed after that "age" that would have triggered the cleanup.

[vidjuheffex]

yeah the laziness I didn't view as an issue, mainly that without new jobs the need for the freed up space also doesn't exist.