tiflow icon indicating copy to clipboard operation
tiflow copied to clipboard

The creation time of the tidb_cdc.syncpoint_v1 table of the secondary cluster does not meet expectations

Open apollodafoni opened this issue 1 year ago • 8 comments

What did you do?

  1. Set the syncpoint interval to 5 minutes, and complete the creation of changefeeds for three clusters at 12:32:30
  2. Primary cluster: tidb_cdc.syncpoint_v1 was created at 04:35:00
  3. Secondary cluster: created at 2024-07-02 04:45:22, and it takes 13 minutes from create changefeed to tidb_cdc.syncpoint_v1 This delay is greater than 2 syncpoint intervals
  4. The ticdc config is as follows bdr-mode = true force-replicate = true enable-sync-point = true sync-point-interval = "5m"

What did you expect to see?

The creation time of the tidb_cdc.syncpoint_v1 table of the secondary cluster within 1~2 syncpoint intervals

What did you see instead?

  • more than 2 syncpoint intervals
  • the changefeed status is warning
  • the changefeed lag is very large

Versions of the cluster

ticdc git_hash 2db092244d18c043f7f8ecc85a598680b33cc324 tidb git_hash 585c631742acb338c9bd0913a89efc3efb5b884a

current status of DM cluster (execute query-status <task-name> in dmctl)

No response

apollodafoni avatar Jul 02 '24 08:07 apollodafoni

/assign @CharlesCheung96

apollodafoni avatar Jul 02 '24 08:07 apollodafoni

/severity major

apollodafoni avatar Jul 02 '24 08:07 apollodafoni

/area ticdc

apollodafoni avatar Jul 02 '24 08:07 apollodafoni

/remove-area dm

apollodafoni avatar Jul 02 '24 08:07 apollodafoni

/label affects-8.2.0

apollodafoni avatar Jul 02 '24 08:07 apollodafoni

@apollodafoni: The label(s) affects-8.2.0 cannot be applied. These labels are supported: duplicate, bug-from-internal-test, bug-from-user, ok-to-test, needs-ok-to-test, affects-5.4, affects-6.1, affects-6.5, affects-7.1, affects-7.5, affects-8.1, affects-8.2, may-affects-5.4, may-affects-6.1, may-affects-6.5, may-affects-7.1, may-affects-7.5, may-affects-8.1, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.5, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.5, needs-cherry-pick-release-8.1, needs-cherry-pick-release-8.2, question, release-blocker, wontfix, MariaDB.

In response to this:

/label affects-8.2.0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot[bot] avatar Jul 02 '24 08:07 ti-chi-bot[bot]

/label affects-8.2

apollodafoni avatar Jul 02 '24 08:07 apollodafoni

[2024/07/02 04:40:08.276 +00:00] [ERROR] [changefeed.go:325] ["an error occurred in Owner"] [namespace=default] [changefeed=cluster-a-cluster-b] [error="[CDC:ErrMySQLTxnError]MySQL txn error: failed to create syncpoint table;: Error 8263 (HY000): The operation is not allowed while the bdr role of this cluster is set to secondary."]

The root cause is that ticdc did not set write source when creating syncpoint table.

CharlesCheung96 avatar Jul 02 '24 08:07 CharlesCheung96