scylla-rust-driver icon indicating copy to clipboard operation
scylla-rust-driver copied to clipboard

Don't use LocalQuorum as default consistency for SimpleStrategy keyspaces

Open cvybhu opened this issue 3 years ago • 4 comments

By default the driver performs each query with the LocalQuorum consistency.

This choice of consistency causes trouble for multi-dc clusters with SimpleStrategy. Imagine a cluster with 3 datacenters (dc1, dc2, dc3), with 3 nodes in each datacenter.

Now let's say there's a keyspace:

CREATE KEYSPACE ks WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 3};

'replication_factor': 3 means that for each partition there are 3 nodes in the whole cluster that are replicas for this partition. Let's say the replicas are: A (in dc1), B (in dc1) and C (in dc2).

Now the user performs a query, and let's say that the load balancing chooses C to be the coordinator. Because the query uses LocalQuorum as the default consistency, the query will fail - there aren't enough nodes in dc2 to fulfill the consistency requirement, there would need to be at least 2.

I think it would be generally better to have the default be Quorum instead of LocalQuorum. It's the safest option. Sadly changing this would count as a breaking change.

cvybhu avatar Mar 16 '23 15:03 cvybhu

Note that we'd like to deprecate SimpleStrategy (see https://github.com/scylladb/scylladb/issues/5224 and others).

mykaul avatar Mar 16 '23 15:03 mykaul

@Lorak-mmk / @wprzytula since we shouldn't support SimpleStartegy with multiDC, can we close this one?

roydahan avatar Jun 13 '24 14:06 roydahan

@Lorak-mmk / @wprzytula since we shouldn't support SimpleStartegy with multiDC, can we close this one?

If the reason for that is that Scylla won't support SimpleStrategy with multiDC, then remember that Cassandra will still support it, and we want to correctly support Cassandra as well. Right?

wprzytula avatar Jun 13 '24 17:06 wprzytula

Yes, but also for Cassandra it's highly advisable not using SimpleStrategy with Multi DC configuration.

roydahan avatar Jun 13 '24 17:06 roydahan

Sadly changing this would count as a breaking change.

This is not really true in the Rust definition of breaking change I think (cc @wprzytula - am I wrong) because it does not change any public API (= it is not possible for some code to stop compiling because of this change). Of course it is a breaking change in a more practical way of defining this term - changing the default consistency could impact user applications.

We could change the default in 2.0, but we should decide if we even should. Imo the default should be suited for the common case, not for each weird corner case, so the argument with multi-DC SimpleStrategy keyspace does not convince me. If we accept that, then the question is what should the default be for more typical cases - LocalQuorum or Quorum? What defaults do other drivers have?

Lorak-mmk avatar May 05 '25 09:05 Lorak-mmk

We should deprecate SimpleStrategy... I think (know) there's an issue for it in Scylla core somewhere.

mykaul avatar May 05 '25 10:05 mykaul

In Rust Driver we don't even use SimpleStrategy in tests, and in my previous comment I said that the default consistency should be suited for more common cases (by which I meant e.g. NTS). So the question to you: what should the default consistency be?

Lorak-mmk avatar May 05 '25 10:05 Lorak-mmk

In Rust Driver we don't even use SimpleStrategy in tests, and in my previous comment I said that the default consistency should be suited for more common cases (by which I meant e.g. NTS). So the question to you: what should the default consistency be?

QUORUM? It can't be ALL and it can't be ONE. Anything in between? I think there's an interesting discussion between LOCAL_Q and regular.

mykaul avatar May 05 '25 11:05 mykaul

There is no way to make default consistency dependent on replication strategy. We can discuss if the default in general should be quorum, or local_quorum. For now it is local_quorum, and if there are arguments for changing that I invite anyone interested to open a new issue about that.

Lorak-mmk avatar Sep 23 '25 10:09 Lorak-mmk