tidb icon indicating copy to clipboard operation
tidb copied to clipboard

Lightning: tidb version number cause parallel import conflict records

Open db-will opened this issue 3 years ago • 0 comments

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

Build tidb binary by yourself, and make tidb shows commit id as tidb release version:

mysql> select tidb_version();
Release Version: bcf61918
...
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306

when you setup parallel import, and run it against the given tidb cluster, it will show duplicate records error.

The issue is because https://github.com/pingcap/tidb/blob/release-6.1/br/pkg/lightning/restore/restore.go#L1592 the version number parsing logic will try to match the first version number it meet in the version string, so the tikv min version will be used by lightning as tidb release version. and then, it will skips the logic due the tidb version not large than 4.0. Such behavior will cause duplicate records error.

We need to fix the tidb version parsing logic: https://github.com/pingcap/tidb/blob/release-6.1/br/pkg/version/version.go#L321 We can display some warning when tidb version number can't be found.

2. What did you expect to see? (Required)

Import success

3. What did you see instead (Required)

import failures due to duplicate records

4. What is your TiDB version? (Required)

custom binary

mysql> select tidb_version();
Release Version: bcf61918
...
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306

db-will avatar Aug 10 '22 02:08 db-will