MicroCeph Remote Replication (3/3): Site Failover/Failback

[UtkarshBhatthere]

Description

This change contains feature work to perform promotion/demotion for ALL pools across the cluster.

Type of change

Please delete options that are not relevant.

• [ ] Bug fix (non-breaking change which fixes an issue)
• [X] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] CleanCode (Code refactor, test updates, does not introduce functional changes)
• [ ] Documentation update (Doc only change)

How Has This Been Tested?

NOTE: All functional changes should accompany corresponding tests (unit tests, functional tests etc).

• [ ] Add Functional Tests
• [ ] Add Unit Tests

Documentation

• [ ] Add command references
• [ ] Add a How-To

Please check that you have:

• [ ] self-reviewed the code in this PR.
• [ ] added code comments, particularly in hard-to-understand areas.
• [ ] updated the user documentation with corresponding changes.
• [ ] added tests to verify effectiveness of this change.