Support dynamically updating `hosts.yaml`

[jaywonchung]

Use case:

• Add a node to hosts.yaml when you get access to more nodes, and the jobs you are currently running will automatically go there.
• Remove a node from hosts.yaml when someone asks you to vacate the node.

Removing a node from hosts.yaml does not kill the job. However, it is guaranteed that the removed node will not be given a new job once it's been removed from hosts.yaml.