is it possible to pause a queue?

[pthieu]

I currently use GCP cloud tasks at my current job and would like to move our queues in-house to simplify our infra.

pg-boss is doing everything we currently need it to do, except there doesn't seem to be a way to pause/unpause the queue. According to the docs, we might be able to do a .deleteQueue(), but as it stands now, we have to delete all the jobs in the table for this to not violate the FK constraint, see this

The unpause/pause is good for us to temporarily stop the queue without losing job data. There have been times when an external service we're calling breaks down and we had to pause a queue to investigate.

Here's a screenshot of GCP's UI for this:

Would this be something that pg-boss will support in the future or is it out of scope by design?

[timgit]

I'm not against this idea. It does seem like it requires quite a bit of work to introduce a queue state and update all APIs and worker interactions. You'd basically want to keep all workers up and running and polling, but have them detect a state change and respond with no jobs.

[xgenvn]

I think it's also possible to add an intermediate queue/worker setup. From front service where we trigger "pause" command, perhaps cancel/remove the task from queue, store original queue name, and add to "paused" queue. Then "unpause" will move to original queue based on the metadata. Maybe we need to take care of race condition while doing so. I checked the pgboss APIs: delete, cancel jobs are available.

@timgit what do you think about this approach?

[timgit]

I would prefer to not do any data movements, since this operation could be long-running based on the queue size.