python3-daemon icon indicating copy to clipboard operation
python3-daemon copied to clipboard

Perhaps another multiple worker daemon example?

Open tomwitt2 opened this issue 5 years ago • 0 comments

First, I wanted to say thank you very much for these examples. They have been invaluable to me. I truly appreciate the code you've provided.

In particular, I found 5-daemon-multiple-worker to be the most helpful. I notice that this example relies on the worker processes to manage their own work loads. I'd be very interested in an example that might show multiple workers with one coordinating master process doling out the incoming work.

My particular use case is to monitor a queue and dispatch work units to workers. For, 5-daemon-multiple-worker, each worker will need to make a connection to the queue and pull the work for it's process. Ideally, a master process might keep track of how many workers are currently working then start a new worker process for incoming units of work as older worker processes finish and die. This way, only the master process is reading off the queue and it would be possible to increase/decrease the number of workers by sending signals to the process to dynamically tailor the work load without restarting the process.

I don't (yet) know how to write such a beast but I'm going to be trying.

Thanks!

tomwitt2 avatar Mar 30 '19 20:03 tomwitt2