Client should not connect to an old primary replica

[lolski]

Description

Under network partitioning scenario, it is possible for a cluster to have two primary replicas. Primary replicas are assigned a term number, therefore it is possible to tell which replica is the newer one out of the two.

However, currently Client Java ignores the term number, and therefore it may first connect to the newer one only then to failover to the older one.

It is important to say that this scenario won't compromise data correctness since the Raft protocol guards against it. However, it'll be more efficient if the term information is used such that the failover always happens from the old to the new primary replica.

Environment

1. OS (where Grakn server runs): Any
2. Grakn version (and platform): 2.0.0-alpha
3. Other environment details: N/A

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Create a cluster of three nodes and wait for a primary replica to be selected by the cluster members
2. Disconnect the network from the primary replica to the other nodes, and wait for the rest of the cluster members to elect a new replica
3. Connect to the new primary using Client Java, by specifying the IP of the new primary replica the secondary replica of which will also be successful
4. Then, connect to the old primary using Client Java by specifying the IP directly, which will be successful

Expected Output

Step #4 should fail. Given that Client Java has already seen the new primary, it should reject the attempt to connect to the old one.

Actual Output

Step #4 succeeded.

[haikalpribadi]

What project should this go into @lolski ? I presume 2.0 Production? I'll move it there for now.