tiflow icon indicating copy to clipboard operation
tiflow copied to clipboard

changefeed resolved ts can not go on after inject network partition between one of ticdc processor and pd leader last for 10mins

Open Lily2025 opened this issue 1 year ago • 5 comments

What did you do?

1、create changefeed 2、run sysbench read_write 3、inject network partition between one of ticdc processor and pd leader last for 10mins image

What did you expect to see?

after clean chaos,changefeed resolved ts can go on

What did you see instead?

changefeed resolved ts can not go on after inject network partition between one of ticdc processor and pd leader last for 10mins

image

Versions of the cluster

./cdc version Release Version: v6.5.7 Git Commit Hash: e9d1010cac9482775062b507c07176aa35e2a67d Git Branch: heads/refs/tags/v6.5.7 UTC Build Time: 2023-12-28 14:08:52 Go Version: go version go1.19.13 linux/amd64 Failpoint Build: false

Lily2025 avatar Jan 02 '24 07:01 Lily2025

/assign lidezhu

Lily2025 avatar Jan 02 '24 07:01 Lily2025

It seems the error is caused by failed to load pd regions for a given span image

lidezhu avatar Jan 02 '24 07:01 lidezhu

/severity major

fubinzh avatar Jan 04 '24 06:01 fubinzh

/severity moderate

lidezhu avatar Jan 05 '24 01:01 lidezhu

/severity-remove major

lidezhu avatar Feb 29 '24 10:02 lidezhu

Fixed by https://github.com/pingcap/tiflow/commit/3bfba30cf45f9f686610f6868bf61ce1bc09bd2b

asddongmen avatar May 21 '24 03:05 asddongmen