Support for failover hosts

[grtjn]

Roxy doesn't support adding failover hosts to forests. For databases that are not spread across a group automatically, one could do:

    <assignment>
      <forest-name>@ml.modules-db</forest-name>
      <host-name>host1</host-name>
      <failover-hosts>
        <failover-host>host2</failover-hosts>
        <failover-host>host3</failover-hosts>
      </failover-hosts>
    </assignment>

For databases with forests-per-host enabled:

    <assignment>
      <forest-name>@ml.content-db</forest-name>
      <failover-hosts count="2"/>
    </assignment>

[grtjn]

Tip: would be nice if not all forests of host1 would failover to host2, but would spread among all other hosts. Perhaps something like failovernr=(hostnr + forestnr + failovercount) mod nrhosts

[grtjn]

Nice to have, pushing away for now..

[RobertSzkutak]

We should review #689 and see if that fulfills the ask in here

[grtjn]

Failover hosts is distinct from forest-replicas. Failover hosts are used for forest data on shared disks (clustered filesystems and such). It was the old approach to provide high availability. Local forest replication was added in later versions of MarkLogic. It has some performance benefits over shared disk approaches, though it comes with a higher disk space footprint.

[grtjn]

It is not used a lot from what I hear, so low prio. I can't recall having heard anyone else ask about it. Would be fun to have it though, even if it were just to enable people to more easily try it out, and compare the options (failover hosts, forest replicas, db replication, flex replication)