Witness evitaDB replica

[novoj]

Viewstamped replication protocol requires commits to be acknowledged by majority of nodes. The problem with evitaDB is that might be quite memory hungry and keeping at least three full replicas might be expensive. Therefore it would be beneficial if some (or all of them except one) are running in "witness" mode.

In this mode:

• replica accepts WAL and participates in transaction commit process
• replica maintains engine state
• replica provides engine state / WAL contents for other replicas / nodes
• replica participates in leader election a. but it cannot become leader b. and it can become leader

Witness replica is not part neither of evita-rw nor evita-ro endpoint slice, i.e. on external traffic cannot reach this replica, but it can communicate with other replicas in this namespace.

If witness replica can become leader it also continuously downloads snapshot backups from S3 system and restores them into a local filesystem to be ready for "activation". When they become leader they simply:

1. load catalogs to active memory (they must have limits for memory large enough)
2. reconcile missing WAL records from other replicas
3. replay missing WAL records
4. become the new leader