foundationdb icon indicating copy to clipboard operation
foundationdb copied to clipboard

Separate critical system metadata from other non-critical ones

Open jzhou77 opened this issue 11 months ago • 0 comments

Critical system metadata, e.g., server list, should always be available. For instance, if RateKeeper can't read server list, it throttles the whole cluster traffic and can cause unavailability.

On the other hand, other non-critical data, e.g., backup, txn samples, can be large. If not carefully configured, e.g., setting transaction sampling ratio to 1, they can hot shard the system keyspace. If the critical metadata happens to be on the same shard, then the database can become unavailable.

The solution to this problem can be solved if we can separate critical system metadata from other non-critical ones, i.e., storing them on different shards, thus reducing the chance of unavailability.

jzhou77 avatar Mar 12 '24 20:03 jzhou77