pax icon indicating copy to clipboard operation
pax copied to clipboard

PullFromQueue timeouts

Open JelleAalbers opened this issue 8 years ago • 0 comments

If the host process of the input process in the standard multiprocessing chain (#439) crashes, or the machine running the process drops of the network, the worker processes hang forever. I can't implement a simple timeout (as I have for the output process), as some some workers never see events when we're running with many more paxes than we need.

We could implement some sort of round-robin distribution of events, rather than just a nondeterministic grabbing from a queue. However, that would require sending acknowledgements to the broker when we're done with events, so it knows we can receive new ones.

JelleAalbers avatar Oct 31 '16 12:10 JelleAalbers