DM will restart dumping while meeting errors in the load phase

[SE-Bin]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a recipe for reproducing the error. If DM meets an error when restoring schema information in the load phase, restarting the task will cause the upstream data to be dumped again.

2. What did you expect to see? If I can fix the errors of schema SQL, it can resume restoring the data after restarting the task in the load phase.

3. What did you see instead? DM will restart the dump phase to dump the data of the upstream. Because there is no checkpoint information in TiDB.

4. Versions of the cluster

   • DM version (run dmctl -V or dm-worker -V or dm-master -V):

     DM 2.0.1
     

   • Upstream MySQL/MariaDB server version:

     MySQL 5.7.30
     

   • Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

     TiDB 4.0.9
     

   • How did you deploy DM: DM-Ansible or manually? I use tiup to deploy DM .

5. current status of DM cluster (execute query-status in dmctl)

Found dmctl newer version:

    The latest version:         v2.0.2
    Local installed version:    v2.0.1
    Update current component:   tiup update dmctl
    Update all components:      tiup update --all

Starting component `dmctl`: /home/tidb/.tiup/components/dmctl/v2.0.1/dmctl/dmctl --master-addr 10.66.50.196:48261 query-status video-dw-task
{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "mysql-video-dw",
                "worker": "dm-10.66.50.196-48262",
                "result": null,
                "relayStatus": null
            },
            "subTaskStatus": [
                {
                    "name": "video-dw-task",
                    "stage": "Paused",
                    "unit": "Load",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 10006,
                                "ErrClass": "database",
                                "ErrScope": "downstream",
                                "ErrLevel": "high",
                                "Message": "run table schema failed - dbfile ./dumped_data.video-dw-task/video_dw.v_video_recommend-schema.sql: execute statement failed: CREATE TABLE `v_video_recommend` (`site` varchar(50) CHARACTER SET utf32 NOT NULL COMMENT '推荐位(首页上部为default-big,default-side,default-side2，其它的取game_id)',`edit_video_ids` varchar(100) DEFAULT NULL COMMENT '编辑推荐',`sequence_video_ids` varchar(100) DEFAULT NULL COMMENT '计算推荐',PRIMARY KEY (`site`)) ENGINE=InnoDB DEFAULT CHARSET=utf8;",
                                "RawCause": "Error 1115: Unknown character set: 'utf32'",
                                "Workaround": ""
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "load": {
                        "finishedBytes": "0",
                        "totalBytes": "369726668001",
                        "progress": "0.00 %",
                        "metaBinlog": "(mysql-bin.045683, 27734088)",
                        "metaBinlogGTID": "00bee0fb-b432-11ea-9a6e-3868dd276fe1:1-10881299,5b4bda08-467b-11eb-85c9-3868dd2d2e28:1-5456180,cd78bb89-85c8-11e9-aa4d-84ad58ec1893:6984057390-7005655412,fd6e4e55-b431-11ea-9f40-3868dd27b129:1-7291943727,006ce21b-b432-11ea-a894-3868dd27ce59:1-11187185"
                    }
                }
            ]
        }
    ]
}

6. Operation logs

   • Please upload dm-worker.log for every DM-worker instance if possible
   • Please upload dm-master.log if possible
   • Other interesting logs
   • Output of dmctl's commands with problems

7. Configuration of the cluster and the task

   • dm-worker.toml for every DM-worker instance if possible
   • dm-master.toml for DM-master if possible
   • task config, like task.yaml if possible
   • inventory.ini if deployed by DM-Ansible

8. Screenshot/exported-PDF of Grafana dashboard or metrics' graph in Prometheus for DM if possible

[lance6716]

This problem is caused by dump unit always returns true in https://github.com/pingcap/dm/blob/50841fb066119153c3b33a92ff9e57959e6ff914/dumpling/dumpling.go#L197-L199

Maybe we should check dump files in local disk