tiflow icon indicating copy to clipboard operation
tiflow copied to clipboard

qps drop 50% when inject ticdc owner io hang

Open Lily2025 opened this issue 9 months ago • 1 comments

What did you do?

1、create changefeed 2、run sysbench subType:"oltp_read_write" tableNum:64 tableSize:70000000 threads:32 3、inject ticdc owner io hang

What did you expect to see?

qps should be normal

What did you see instead?

qps drop 50% when inject ticdc owner io hang image

image

Versions of the cluster

./cdc version Release Version: v8.1.0 Git Commit Hash: 38eb040812f817df410abe870952629df60fd01f Git Branch: HEAD UTC Build Time: 2024-04-30 06:27:14 Go Version: go version go1.21.4 linux/amd64 Failpoint Build: false

current status of DM cluster (execute query-status <task-name> in dmctl)

No response

Lily2025 avatar May 11 '24 03:05 Lily2025

/remove-area dm /area ticdc /type enhancement /assign asddongmen

Lily2025 avatar May 11 '24 03:05 Lily2025

I will do more investigation on this issue and provide a concrete reason within two days.

asddongmen avatar May 15 '24 10:05 asddongmen

The HA test results from last week (2024/5/15 - 2024/5/19, conducted 3 times) indicate that the CDC does not impact upstream performance. If the issue reoccurs, I will examined further. @Lily2025 cc @flowbehappy

asddongmen avatar May 20 '24 02:05 asddongmen