feat: support to force send non-idempotent write when doing duplication
What problem does this PR solve?
#1907
What is changed and how does it work?
- Change the judgment logic in the writing process, and configure it to allow non-idempotent operations in the master cluster.
- Modify the logic of the specified mutation committer in prepare_list
- Add configuration items and add them to the replica side to modify them through the server-config of admin-cli.
- Add the parsing logic after the backup cluster receives the packaged RPC of duplication.
- Add the logic for the master cluster to obtain relevant non-idempotent RPC from the plog
- Add the metric index of dup non-idempotent write QPS
Tests
- Unit test
- Manual test (add detailed scripts or steps below)
when user want to force dup non-idempotent mutations. Type following command in admin-cli:
server-config replica force_send_no_idempotent_when_duplication set true
This PR's base function is in line with expectations , however I was requested by business department to add more matrix for surveillance system about this feature. I will update this PR in recently days.
Now , when user config force_send_no_idempotent_when_duplication to true . We can dup non-idempotent write to backup cluster , and we can verify the data consistency by check the number rows of pegasus app.
app in master-cluster
app in backup-cluster
And because the retries for non-idempotent writes will make influence of the consistency between two clusters. So we should show the metric like following picture:
show in backup cluster: