Easy to generate conflicting versions

[dalvizu]

To reproduce:

• Schema starts at version '100'
• Developer A, using 'git flow' workflow (http://nvie.com/posts/a-successful-git-branching-model/) works on a feature branch A, and needs to make a migration. Chooses version 101
• Developer B, also working on a feature branch B, also needs to make a migration. Chooses version 101
• Both merge to develop branch

Expected results:

First developer will see their updates apply, second will never apply

[mgobec]

I would say that this is a developer issue. No way we can stop developers from doing s**t like that with this framework. Thats why I always rebase from target branch before making a PR. You guys should fire the one merging the code without rebase 😄

[dalvizu]

Okay so help me with my case so I don't get fired ;)

I'm building a developer pipeline - I'm allowing them to develop migrations on their feature branches. I'm wrapping this in a DSL so they can only do 'safe' operations. I generate the migrations for them, and use an index of (last migration+1) in the new migration.

When they want to push to production, they first merge their branch to master, this creates a new build which deploys to a test environment first, where we bundle their DSL to a release artifact, and run the migration engine on this DSL (which generates a MigrationResources) on this test environment.

We then run automated tests, and do manual QA, and if all looks good we go to production with a click of a button on a CD pipeline (in our case, gitlab CI).

Developer B, before they do their merge to master, could do a rebase and will see no issue - either a merge conflict or otherwise. It's working on his laptop.

I could, I suppose, add a check myself in the pipeline: look at these migrations at the time of apply, look at what's in the environment already, and if there's some other migration with the same version there - error out. Or if I'm trying to apply the same version twice, error out. I now require developer B to go back (after merge) and change their version, and update their development environment to match.

That's not an ideal experience, in my opinion.

No way we can stop developers from doing s**t like that with this framework

There is a way to stop this - use active record's approach - use timestamp to get a unique and self-documenting version everytime

[mgobec]

Are you then proposing to have a timestamp for "version"? I'm not sure this would solve the issue since database migration development should be synced between developers. You should be aware of the version you are upgrading from, thus rebase is much needed. The solution you are proposing could lead to unwanted database state. Just my two cents (and some experience).

[mgobec]

I'll keep it open to keep the conversation for now.

[dalvizu]

Are you then proposing to have a timestamp for "version"?

Yup! I think we're on the same page - I want to help my devs stay in sync with database migrations, and I think a unique timestamp version would help. The use-case I would want to support is:

• Developer A and B start feature branch
• Developer A realizes need for data update, starts one, version X
• Developer B realizes need for data update, starts her own, version Y
• Developer A + B are in scrum standup, aware of each-others changes, coordinate
• Developer B task was much smaller - ready to deploy, starts pipeline
• Pipeline merges to master, applies updates to test, database now has updates Y
• Developer A pulls developer B's change, runs migrations, has both version X and Y applied
• Developer A ready to push - starts pipeline
• Pipeline merges to master, applies updates to test, database now has updates X and Y applied

With timestamp, I believe the only coordination necessary is 'pull before you push', much like application changes.

With integer version (along with #33), X == Y at first, and they need to figure out which actual number to use themselves, which is going to be dependent on who deploys to the shared environment first. In order to share changes, they need to do some manual running of the changes on their local environments in order to pick them up.

It's doable, but manual and not ideal. And it becomes more complicated if there's more than two people.