poolboy
poolboy copied to clipboard
Local Pool (manager) with Remote Workers + Configurable Worker collection
This PR should really be 2 as the title suggest. However, it was easier for me the to create it as single one. - Apologies! - I just wanted to get your thoughts on the changes first and find out whether you see these extensions useful. - I'm happy to take this further, e.g.: split the PR, etc.
Remote Workers
I use poolboy
at work and needed to create a pool of processes similar to the rpc
module implementation in Erlang/OTP. - rpc
(or rex
) is a single process, I needed a pool of them. - This led me to extend poolboy
, so that (local) workers could be started by a supervisor on a remote node.
Worker Collection
In situations, where the worker is non-blocking, the pool performs better if the checkout
, checkin
mechanism is not used. However, it's still a good idea to distribute the work-load between workers, e.g.: simply relying on random distribution. - This led me to consider random-access
data-structures, like array
and tuple
for the collection of workers. - I've added a module: poolboy_collection
that encapsulates the data-structure, in which the workers are stored.
There's a lot of changes here and I'm not sure when I'll have time to review this :/ I will try to find some time though.
Hi @Vagabond, you are right this has become a mess. Sorry about that. I would be happy to rework these changes into manageable chunks (PRs). Please let me know if that would actually work for you ;)
Hi again @Vagabond, I've started on spitting this PR into manageable pieces.
Here's the first chunk: https://github.com/devinus/poolboy/pull/140 - Still somewhat large but I think it's manageable.
Thanks!