securedrop-client icon indicating copy to clipboard operation
securedrop-client copied to clipboard

Do we need a way to pause queues outside of the queues themselves?

Open sssoleileraaa opened this issue 4 years ago • 0 comments

Description

Either remove PauseQueueJob since it's currently unnecessary or re-implement having the queue manager pause both queues when a RequestTimeoutError is encountered.

I think this should be discussed more along with https://docs.google.com/document/d/1-Y6eK0q6Bpxe4IcGFdyVZ2rqXxRqAFCbiiapEWqfUuE/edit?usp=sharing, which proposes that we silently pause the queues and rely on our sync/health checker to report network errors.

So far I'm in favor of removing PauseQueueJob.

Background

PauseQueueJob was implemented because we wanted a way to pause both queues from the queue manager. After creating PauseQueueJob, a lot of work was done on the queue, including adding job priorities and a retry link to resume the queue. During this active queue development time, we discussed how it would be perfectly acceptable, and simpler, if queues were responsible for pausing themselves. Also, we discussed how we should use put_nowait internally, inside the processing loop of the RunnableQueue, instead of relying on the queue manager to enqueue PauseQueueJob before exiting. This was to ensure that we paused the queue right away.

At the time, I didn't realize that we could just remove PauseQueueJob all together since its original purpose was for allowing the queue manager to pause both queues.

sssoleileraaa avatar Feb 13 '20 03:02 sssoleileraaa