loader, syncer: support create/drop view

[lance6716]

What problem does this PR solve?

close #1300

What is changed and how it works?

support dumpling's view file in load unit, support sync view as well.

sync of view didn't need a sharding sync lock, that is to say we could directly execute CREATE VIEW IF NOT EXISTS / DROP VIEW IF EXISTS to downstream when any shard send it

Check List

Tests

• Unit test
• Integration test

Code changes

Side effects

Related changes

• Need to cherry-pick to the release branch
• Need to be included in the release note

[lance6716]

waiting https://github.com/pingcap/dm/pull/1309

[lance6716]

binlog-filter in tidb-tools doesn't prepared for VIEW statements, will file PR later

[lance6716]

waiting https://github.com/pingcap/tidb-tools/pull/405 (but I think it's OK to merge this PR before tidb-tools. In days before v2.0.1 release will update dependencies as well)

[lance6716]

and supporting VIEW may block user's migration, such as need handle VIEW as unsynced shard DDL? should we defaultly disable this 🤔 @csuzhangxc @GMHDBJD

[csuzhangxc]

and supporting VIEW may block user's migration, such as need handle VIEW as unsynced shard DDL? should we defaultly disable this 🤔 @csuzhangxc @GMHDBJD

Disable it in this PR, and try to support view in shard mode later?

[lance6716]

remember to revert VIEW part of https://github.com/pingcap/dm/pull/1331

[lance6716]

updated code and PR description, PTAL @lichunzhu @GMHDBJD @csuzhangxc

[lance6716]

PTAL @GMHDBJD

[lance6716]

data race recorded in https://github.com/pingcap/dm/pull/1351

[2021-01-07T04:24:29.483Z] WARNING: DATA RACE
[2021-01-07T04:24:29.483Z] Write at 0x00c0002da4f0 by goroutine 194:
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/pkg/gtid.(*MySQLGTIDSet).Set()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/pkg/gtid/gtid.go:105 +0xb5
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/relay.(*Relay).handleEvents()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/relay/relay.go:474 +0x1217
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/relay.(*Relay).process()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/relay/relay.go:257 +0x742
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/relay.(*Relay).Process()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/relay/relay.go:173 +0xb1
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/dm/worker.(*realRelayHolder).run()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/relay.go:141 +0x1e8
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start.func1()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/relay.go:117 +0x8a
[2021-01-07T04:24:29.483Z] 
[2021-01-07T04:24:29.483Z] Previous read at 0x00c0002da4f0 by goroutine 196:
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/pkg/gtid.(*MySQLGTIDSet).Clone()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/pkg/gtid/gtid.go:159 +0x8b
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/relay.(*LocalMeta).GTID()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/relay/meta.go:352 +0x101
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/relay.(*Relay).Status()
[2021-01-07T04:24:29.483Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/relay/relay.go:763 +0x213
[2021-01-07T04:24:29.483Z]   github.com/pingcap/dm/dm/worker.(*realRelayHolder).Status()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/relay.go:162 +0x1a3
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*realTaskStatusChecker).checkRelayStatus()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/task_checker.go:305 +0x156
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*realTaskStatusChecker).check()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/task_checker.go:410 +0xfa
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*realTaskStatusChecker).run()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/task_checker.go:216 +0x2ba
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*realTaskStatusChecker).Start.func1()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/task_checker.go:177 +0x8a
[2021-01-07T04:24:29.484Z] 
[2021-01-07T04:24:29.484Z] Goroutine 194 (running) created at:
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*realRelayHolder).Start()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/relay.go:115 +0x85
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*Worker).Start()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/worker.go:126 +0x772
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*Server).startWorker.func1()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/server.go:591 +0x59
[2021-01-07T04:24:29.484Z] 
[2021-01-07T04:24:29.484Z] Goroutine 196 (running) created at:
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*realTaskStatusChecker).Start()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/task_checker.go:175 +0x85
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*Worker).Start()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/worker.go:134 +0x6c8
[2021-01-07T04:24:29.484Z]   github.com/pingcap/dm/dm/worker.(*Server).startWorker.func1()
[2021-01-07T04:24:29.484Z]       /home/jenkins/agent/workspace/dm_ghpr_test/go/src/github.com/pingcap/dm/dm/worker/server.go:591 +0x59
[2021-01-07T04:24:29.484Z] ==================

[ti-chi-bot]

@lance6716: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[lance6716]

/hold