FgForrest
Distributed database support
Database will internally use viewstamped replication by (Viewstamped Replication Revisited paper) - it goes along the principles we favour in the evitaDB philosophy.
Information about cluster topology are versioned and stored in the engine state. Engine state progresses only on a leader node.
evitaDB will define generic VSR API that will initially contain only single K8S implementation.
- leader will maintain leased lock to keep its leadership
- any replica can initiate leader takeover if it's able to lease shared lock
- epoch will be added to transactions to prevent write splits
- there will be test suite for cluster reconfiguration scenarios using https://github.com/fabric8io/kubernetes-client/blob/main/doc/kube-api-test.md
- we need to distinguish startup and readiness probe
Clients will use two local DNS endpoints: evita-rw and evita-ro. Clients opening read-only sessions will use evita-ro, clients opening read-write sessions must use evita-rw endpoint. If the evita-rw endpoint connection is terminated, they need to renew the connection, if attempt to creating R/W session on that endpoint fails, they need to reopen HTTP connection and retry.
When leader fails to renew lease or crashes (and any replica seeing this initiate view change) it is automatically removed from evita-rw endpoint slice. New leader after state reconciliation patches evita-rw endpoint slice with its own IP address. So there could be a limited period when the cluster is in read-only mode.
Resources
The issue is just a setup to collect interesting information:
- https://tikv.github.io/deep-dive-tikv/distributed-transaction/distributed-algorithms.html
- https://decentralizedthoughts.github.io/2020-12-12-raft-liveness-full-omission/
- https://omnipaxos.com/blog/how-omnipaxos-handles-partial-connectivity-and-why-other-protocols-cant/
Ctrl node protocol recovery paper:
- https://ramalagappan.github.io/pdfs/papers/par.pdf
Viewstamped replication:
- https://youtu.be/Wii1LX_ltIs?si=IuDVQrbd9Hcl633c
K8S operator:
- https://github.com/operator-framework/java-operator-sdk
