hammer icon indicating copy to clipboard operation
hammer copied to clipboard

Migrate from spansql to memefish

Open morikuni opened this issue 1 year ago • 4 comments

fix: https://github.com/daichirata/hammer/issues/65

There are some DDLs that cannot be parsed by spansql, so I migrated to memefish while trying to maintain as much compatibility as possible. Due to differences in SQL formatting between spansql and memefish, there are significant diffs in the outputs. However, I believe the functionality remains unchanged.

spqnsqlではパースできないDDLがあるので、できるだけ互換性を保ったままmemefishに移行した。spansqlとmemefishのSQLのフォーマットの違いにより、diffに差分が大きく出ているが、動作は変わっていないと思う。

morikuni avatar Dec 17 '24 06:12 morikuni

@morikuni I tested it with actual data and noticed the following issue:

When the order of an index or primary key is ASC, it is omitted from the DDL returned by the Spanner API. As a result, if the user explicitly specified ASC, a new difference is detected.

Could you check this?

		{
			from: `
CREATE TABLE t1 (
  t1_1 INT64 NOT NULL,
  t1_2 INT64 NOT NULL,
) PRIMARY KEY(t1_1, t1_2);
`,
			to: `
CREATE TABLE t1 (
  t1_1 INT64 NOT NULL,
  t1_2 INT64 NOT NULL,
) PRIMARY KEY(t1_1, t1_2 ASC);
`,
			expected: []string{},
		},

		{
			from: `
CREATE TABLE t1 (
  t1_1 INT64 NOT NULL,
  t1_2 INT64 NOT NULL,
) PRIMARY KEY(t1_1, t1_2);
CREATE INDEX idx_t1 ON t1(t1_1, t1_2);
`,
			to: `
CREATE TABLE t1 (
  t1_1 INT64 NOT NULL,
  t1_2 INT64 NOT NULL,
) PRIMARY KEY(t1_1, t1_2);
CREATE INDEX idx_t1 ON t1(t1_1, t1_2 ASC);
`,
			expected: []string{},
		},
# memefish
&ast.IndexKey{DirPos:-1, Name:&ast.Ident{NamePos:79, NameEnd:83, Name:"t1_1"}, Dir:""}
&ast.IndexKey{DirPos:90, Name:&ast.Ident{NamePos:85, NameEnd:89, Name:"t1_2"}, Dir:"ASC"}

&ast.IndexKey{DirPos:-1, Name:&ast.Ident{NamePos:79, NameEnd:83, Name:"t1_1"}, Dir:""}
&ast.IndexKey{DirPos:-1, Name:&ast.Ident{NamePos:85, NameEnd:89, Name:"t1_2"}, Dir:""}

# spansql
[]spansql.KeyPart{spansql.KeyPart{Column:"t1_1", Desc:false}, spansql.KeyPart{Column:"t1_2", Desc:false}}
[]spansql.KeyPart{spansql.KeyPart{Column:"t1_1", Desc:false}, spansql.KeyPart{Column:"t1_2", Desc:false}}

daichirata avatar Feb 05 '25 09:02 daichirata

@daichirata Fixed the problem you mentioned by considering index without direction is ASC.

morikuni avatar Feb 06 '25 13:02 morikuni

@morikuni Thank you for the prompt correction! There is a similar issue with the primary key, so could you please fix it in the same way?

daichirata avatar Feb 07 '25 04:02 daichirata

@daichirata Fixed also for primary key!

morikuni avatar Feb 07 '25 05:02 morikuni

@morikuni We're still running tests, but I'll proceed with the merge for now. Thanks!

daichirata avatar Apr 15 '25 07:04 daichirata

@apstndb Thanks for the reassuring support!

daichirata avatar Apr 15 '25 07:04 daichirata