DM-DIFFERENT_SCHEMA_INCREMENTAL `"result": true count: 2 != expected: 3`

[lance6716]

Which jobs are flaking?

DM-DIFFERENT_SCHEMA_INCREMENTAL

Which test(s) are flaking?

DM-IT shardddl_optimistic

Jenkins logs or GitHub Actions link

https://ci2.pingcap.net/blue/organizations/jenkins/dm_ghpr_integration_test/detail/dm_ghpr_integration_test/8863/pipeline

[2022-08-12T09:09:39.146Z] dmctl test cmd: "resume-task test"

[2022-08-12T09:09:39.402Z] command: resume-task test "result": true count: 2 != expected: 3

[2022-08-12T09:09:39.403Z] {

[2022-08-12T09:09:39.403Z]     "op": "Resume",

[2022-08-12T09:09:39.403Z]     "result": true,

[2022-08-12T09:09:39.403Z]     "msg": "",

[2022-08-12T09:09:39.403Z]     "sources": [

[2022-08-12T09:09:39.403Z]         {

[2022-08-12T09:09:39.403Z]             "result": true,

[2022-08-12T09:09:39.403Z]             "msg": "",

[2022-08-12T09:09:39.403Z]             "source": "mysql-replica-01",

[2022-08-12T09:09:39.403Z]             "worker": "worker1"

[2022-08-12T09:09:39.403Z]         },

[2022-08-12T09:09:39.403Z]         {

[2022-08-12T09:09:39.403Z]             "result": false,

[2022-08-12T09:09:39.403Z]             "msg": "[code=38032:class=dm-master:scope=internal:level=high], Message: some error occurs in dm-worker: ErrCode:36027 ErrClass:\"sync-unit\" ErrScope:\"internal\" ErrLevel:\"high\" Message:\"startLocation: [position: (dm-integration-test-go1180-8863-0pw6v-f7rdt-bin|000001.000001, 8547), gtid-set: 457ca6f6-1a1d-11ed-a841-aeaae3c93fd8:1-34], endLocation: [position: (dm-integration-test-go1180-8863-0pw6v-f7rdt-bin|000001.000001, 8591), gtid-set: 457ca6f6-1a1d-11ed-a841-aeaae3c93fd8:1-34]: gen insert sqls failed, sourceTable: `shardddl1`.`tb1`, targetTable: `shardddl`.`tb`: Column count doesn't match value count: 3 (columns) vs 2 (values)\" Workaround:\"Please check the log files to see if a related DDL has been skipped, and you could use `operate-schema` to get and set the table structure.\" , Workaround: Please execute `query-status` to check status.",

[2022-08-12T09:09:39.403Z]             "source": "mysql-replica-02",

[2022-08-12T09:09:39.403Z]             "worker": "worker2"

[2022-08-12T09:09:39.403Z]         }

[2022-08-12T09:09:39.403Z]     ]

[2022-08-12T09:09:39.403Z] }

Anything else we need to know

• Does this test exist for other branches as well?

• Has there been a high frequency of failure lately?

[lance6716]

I guess that resume-task is not exclusive with query-status, and query-status will see old status