Ensure schema agreement in backup

[Michal-Leszczynski]

Right now Scylla Manager awaits schema agreement before taking a snapshot. This increases the probability of schema agreement in the snapshot to almost 100%, but unfortunately schema change can still happen in between those two operations.

In order to ensure schema agreement in the snapshot, we can:

• await schema agreement and fetch current schema version
• take a snapshot
• again, repeat schema agreement and fetch new schema version
• compare schema versions:
  • schema versions are the same: we are sure that snapshot has schema agreement
  • schema versions are different: retake snapshot or end backup run with an error and retry later

Having different schema versions is highly unlikely to happen, so this wouldn't add any time complexity into the backup procedure. On the other hand, this would give us really important guarantee, which could improve the way in which SM restores backed-up schema.

[Michal-Leszczynski]

Actually, awaiting schema agreement is only possible when SM has cluster CQL credentials - so this idea is only feasible for this scenario. Unless we can require to always have CQL credentials, we can't guarantee schema agreement in backup.