dumpling icon indicating copy to clipboard operation
dumpling copied to clipboard

Write more context in metadata (such as SQL Mode)

Open lance6716 opened this issue 5 years ago • 3 comments

Feature Request

Is your feature request related to a problem? Please describe:

mydumper's metadata has some not so good design, for example, write an empty GTID: when not enable GTID.

and dumpling has diverged a little in https://github.com/pingcap/dumpling/pull/146 . so maybe we could record more context in metadata

Describe the feature you'd like:

SQL Mode (mainly ANSI_QUOTES) is needed to correctly parse dump files, so it should be written somewhere

for example

Started dump at: 2020-08-26 17:54:52
SHOW VARIABLES:
	sql_mode: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

SHOW MASTER STATUS:
	Log: mysql-bin.000002
	Pos: 194
	GTID:09bec856-ba95-11ea-850a-58f2b4af5188:1-103

SHOW MASTER STATUS: /* AFTER CONNECTION POOL ESTABLISHED */
	Log: mysql-bin.000002
	Pos: 199
	GTID:09bec856-ba95-11ea-850a-58f2b4af5188:1-103

Finished dump at: 2020-08-26 17:54:52

Describe alternatives you've considered:

currently DM will query SQL Mode from upstream, so it's OK not add this feature. But if user changes SQL Mode after dump it will fail.

and maybe dumpling could use a default context to dump SQLs.

Teachability, Documentation, Adoption, Optimization:

lance6716 avatar Oct 09 '20 07:10 lance6716

sql_mode is a global/session variable, it may changed easily. I think we can not truly tell what sql_mode is when the table created. so record the sql_mode for current snapshot is meaningless?

3pointer avatar Oct 12 '20 08:10 3pointer

sql_mode take effect when dumpling read, not when table create.

dumpling would use all global variables as its session variables (if not configurated), and in dump process session variables are not changed by iteslf (and can't be changed from outside?). so I think it would represent dump file's context

lance6716 avatar Oct 12 '20 08:10 lance6716

and charset if https://github.com/pingcap/tidb/pull/20409/files

lance6716 avatar Oct 12 '20 09:10 lance6716