Add agent/machine at runtime to JobCluster

[anishsharma]

My requirement is to add a new node/agent/machine to existing cluster when the client program is already running. I am aware that their is allocate_node() but that didn't serve my purpose as client is already running. Also I don't want to add "*" when creating JobCluster because it will cause heavy data traffic initially at our data center and I don't want my client program to ping each and every machine at data center. Just the ones which are currency running dispynode should be identified at runtime. Is there a way I can achieve this in current version of dispy ?

[pgiri]

Until now dispy always sent broadcast information to get all available nodes. I just committed a patch that doesn't do this. You can get this now from github or wait for next release (in couple of days). With the new change, you can list all nodes that you want to use in 'nodes' parameter to cluster.

There are other ways you can control which nodes are used (e.g., you can subclass NodeAllocate class and override allocate method), or use secret argument with nodes so the client will only use nodes that use same secret.

[UnitedMarsupials-zz]

@anishsharma, nodes broadcast their availability to everyone, who'd listen. If anything, we had to implement "secrets" to keep different computer-clusters (Dev, QA, Production) from mixing with each other (each cluster's "secret" contains its designation, among other things).

You can also already add/remove nodes on the fly by accessing Dispy client's web-interface -- not that I'd recommend you use it, because "out of the box" it has no access-control...

Maybe, @pgiri can make the same add/remove functionality available from command-line -- so that an operator could could modify the cluster(s) at run-time without using the HTTP-server...