SQL-agnostic version of migrator

[fabiante]

Hi,

I really like migrator! Overall, it seems like the simplest and versatile migration library in Go I have seen so far.

I have a question: Is there any intend to abstract away the dependency on the sql package? I have a Node.js background and I loved working with https://github.com/sequelize/umzug because it enabled me to store the migration state in a JSON file and I could essentially write migrations that aren't even related to SQL.

Thanks in advance and stay safe!

[lopezator]

Hello, thank you for your words!

This is a very interesting concept.

I would appreciate if you could elaborate a bit more with some usecase examples. How can this be useful for other go users?

Somewhat related with:

https://github.com/lopezator/migrator/pull/32

(Excuse the spanish comments there, but the op explanation and code samples speak for theirselves)

CC/ @pablote

[fabiante]

Hi,

Well, I'd like to create some migrations that involve calling web APIs and setting up an SQLite database (without any fancy ORM tools). Historically I have been able to implement those with Umzug.

I think I was kind of imagining a library that allows me to write such migrations in Go Code (unlike loading SQL files from the FS) and therefore embedd them into a binary. Using such a system would also require to be able to supply the migration log from a different source than a database.

[lopezator]

Thank you for explaining!

With migrator you can already write migrations using:

• go code
• sql (both as string into go code or using separate files, leaving the door open to use 3rd party tools to embed those sql files into the go binary, like https://github.com/mjibson/esc).
• both mixed

In theory nothing stop you from including api calls into your migrations.

I haven't tested, but SQLite should also work well.

What is true right now is that you have to use something compatible with stdlib database/sql and use the same system for the migration log.

I'll investigate this further along with the tool you mentioned, because I think this might be interesting.

Thank you again for raising this!

[fabiante]

Hi, thanks for the reply.

Right now I ended up building a very basic migration system myself. I simply using a slice of "singleton structs" which serve as one migration each.

While doing so I realized that the basic logic of a migration manager isnt as difficult as I thought.

I think that decoupling the migragion log source from the migration source is worth the effort. Same for supporting non-sql migrations.

[lopezator]

I will give this a chance!

I think I could implement a storage interface somehow and pass into into migrator as an option, trying to avoid breaking current users as much as possible.

Something similar to:

https://github.com/sequelize/umzug/blob/master/src/storage/contract.ts

[fabiante]

Nice! If you did that then the old mechanism would have to be refactored to implement that interface. In addition a simple JSON file powered storage would be nice. Depending on how you are planning to implement this, I'd help with implementing either of them, shouldn't be that hard.

[lopezator]

Yup, I just need time, and try to find a way to break the users as less as possible.

I also think the memory storage could help with the testing.

I will ping you if I start with something.

Thank you!