Replace ScyllaDB nodes without affecting the availablility (create a new node first)

[tnozicka]

ScyllaDB support a graceful replace node procedure where a new node is spun up before the old one is decommissioned.

https://opensource.docs.scylladb.com/stable/operating-scylla/procedures/cluster-management/replace-running-node.html#add-a-new-node-to-the-cluster-and-then-decommission-the-old-node

This is incompatible with using StatefulSets where each replica has identity / ordinal and there can always be just one instance of the ordinal.