Investigate effect of asymmetrical network partitions

[joshua-g]

We ran into a situation with our c5 raft implementation, in which a single node was able to disrupt the availability of the rest of the cluster.

In this situation, an asymmetrical network partition occurred: a server was able to successfully send RPCs to other servers in the cluster, but it couldn't receive the replies back; nor could it receive any RPCs coming from other servers. What happened was that the partitioned server began incrementing its term and initiating elections endlessly.

It could be good to investigate how hoverbear/raft handles this scenario.