tiflow
tiflow copied to clipboard
One of TiKV min resolved lag is 3 hours during sysbench prepare
What did you do?
- Test-infra case, 1 changefeed with sink MySQL, base image is alpine.
- workload: sysbench oltp_write_only prepare --tables=128 --table-size=10000000 --time=86400 --debug=true --threads=128 (UTC 2022-05-18T07:20:20 to 2022-05-19T07:30:20) (Beijing 2022-05-18T15:20:20 to 2022-05-19T15:30:20)
- sysbench oltp_write_only run --tables=128 --table-size=10000000 --time=86400 --debug=true --threads=128 (UTC 2022-05-19T07:30:20 to 2022/05/20 07:30:23) (Beijing 2022-05-19T15:30:20 to 2022/05/20 15:30:23)
What did you expect to see?
No response
What did you see instead?
-
TiKV-1 min resolved lag is 3 hours during sysbench prepare, other TiKV min resolved lag is several seconds.

-
Only TiCDC-1 checkpoint lag is 3 hours, while other TiCDC checkpoint lag is < 10 min.

-
TiDB:

Versions of the cluster
Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):
(paste TiDB cluster version here)
v6.1.0-nightly 2022-05-16
Upstream TiKV version (execute tikv-server --version):
(paste TiKV version here)
Release Version: 6.1.0-alpha
Edition: Community
Git Commit Hash: 748f8a95411fc95e05ebc039d233cdfa04e4acb9
Git Commit Branch: heads/refs/tags/v6.1.0-nightly
UTC Build Time: 2022-05-16 07:30:24
Rust Version: rustc 1.60.0-nightly (1e12aef3f 2022-02-13)
Enable Features: jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
TiCDC version (execute cdc version):
(paste TiCDC version here)
Release Version: v6.1.0-nightly
Git Commit Hash: 4a992703a8bd67d173ff4a9d3d9ad8fdda189432
Git Branch: heads/refs/tags/v6.1.0-nightly
UTC Build Time: 2022-05-16 07:29:46
Go Version: go version go1.18 linux/amd64
Failpoint Build: false