dist_test Add labels to test slaves

One label can be os type: ubuntu14, ubuntu16, windows, centos65, etc.

Enable grind to select test slaves based on specified label matching patterns.

Oct 24 '16 18:10 jzhuge

We'd also have to coordinate this with the autoscaler, so it spawns more executors that fit that label.

On the execution side, we could possibly use a beanstalk tube per label.

This might be too much for our simple scheduling system though. Robert/Haibo/Daniel did grind-on-YARN for a hackathon, maybe we should productionize that.

Oct 24 '16 20:10 umbrant

Another option would be docker-in-docker, so that you can specify a particular docker container to run your job in (but still a single pool of executors). Probably more efficient and easy to expand to large numbers of different slave types.

Oct 24 '16 21:10 toddlipcon

+1 for utilizing Docker images. It will be very effective.

We have also a distributed testing tool that is very similar to dist_test, and using Docker images for setting up workers easily (and using Docker Swarm/Kubernetes for scaling) https://github.com/osrg/namazu-swarm

Oct 28 '16 05:10 AkihiroSuda