Support for Spanner DML

[lc-jordanfelix]

Is your feature request related to a problem? Please describe. We would like to be able to load non-transactional data using this tool.

Describe the solution you'd like Running DML statements as well as DDL statements.

Describe alternatives you've considered Using a separate tool. It would be much more useful for us to be able to run both DML and DDL in a migration-like manner using one tool.

Additional context Syntax error on line 1, column 1: Encountered \'INSERT\' while parsing: ddl_statement\nSyntax error on line 1, column 1: Expecting \'EOF\' but found \'INSERT\'

Syntax error on line 1, column 1: Encountered \'UPDATE\' while parsing: ddl_statement\nSyntax error on line 1, column 1: Expecting \'EOF\' but found \'UPDATE\'

We see that these are coming from "cloud.google.com/go/spanner/admin/database/apiv1".DatabaseAdminClient.UpdateDatabaseDdl(...) in migrate/database/spanner/spanner.go and that there may not be support for DML in the same google library, but we hope you might have a solution.

Thanks!

[lc-jordanfelix]

For anyone interested in this as well, we currently are using this script as an alternative for our Spanner DML files. You can find it in our public repo.

We are improving it as we go.

[juztin]

Oh man, just stumbled across this after getting everything all integrated and working within a pipeline. Is it possible to add an asterisk to "Spanner" within the README, stating that it's only partially supported? Not being able to run dml statements is kind of a big deal.

[dhui]

If anyone would like to add DML support for spanner, I'd be happy to review it.

The PR would need to have tests and be backwards compatible. e.g. shouldn't break existing users of the migrate spanner db driver

[jorygeerts]

@dhui Would an approach where each migration file may contain either DDL or DML but not both be acceptable?

If yes, what approach of detecting which a given migration file contains would you prefer, parsing the contents, or adding some kind of "marker" to the filename in case of DML (not required for DDL, to keep things backwards compatible)? Or something else I haven't thought of. :)

[plemanach]

Hi @dhui, I created this PR that add DML for spanner, let me know your thoughts. Thanks, Patrice

[afarbos]

I /fresh created a new PR that can be merged following all your guidelines @dhui, all lines added are tested and it's fully backward compatible with no change from the user pov.