Migrations organised by year and date not working on multiple different namespaces

[JBx0]

Bug Report

Q	A
BC Break	yes
Version	3.1.1

Summary

We have a bundle that holds migrations in our Application we access those migrations and want them to be execute int he order of year and month ideally also day but ok there is no option for it.

In general the sorting looks at the namespace first and afterwards at the date

Current behavior

Upon the migrate command the migrations are first sorted by namespace and afterwards by year and month

How to reproduce

Have a Bundle with Migrations + an application executing those and it's own.

doctrine_migrations:
    organize_migrations: BY_YEAR_AND_MONTH
    migrations_paths:
        'Bundle\DoctrineMigrations': '%kernel.project_dir%/vendor/Bundle-Vendor/custom-bundle/migrations/'
        'DoctrineMigrations': '%kernel.project_dir%/migrations'

DoctrineMigrations\Version20210305101226 vs Bundle\DoctrineMigrations\Version20200110165135

bin/console doctrine:migrations:migrate latest --dry-run --verbose
Xdebug: [Config] The setting 'xdebug.remote_enable' has been renamed, see the upgrading guide at https://xdebug.org/docs/upgrade_guide#changed-xdebug.remote_enable (See: https://xdebug.org/docs/errors#CFG-C-CHANGED)
  [notice] Migrating (dry-run) up to Bundle\DoctrineMigrations\Version20210331160220
  [info] ++ migrating DoctrineMigrations\Version20210305101226
  [info] Migration DoctrineMigrations\Version20210305101226 migrated (took 19.7ms, used 22M memory)
  [info] ++ migrating DoctrineMigrations\Version20210311163822
  [info] Migration DoctrineMigrations\Version20210311163822 migrated (took 0.2ms, used 22M memory)
  [info] ++ migrating DoctrineMigrations\Version20210318104805 
  [info] Migration DoctrineMigrations\Version20210318104805 migrated (took 0.2ms, used 22M memory)
  [info] ++ migrating DoctrineMigrations\Version20210401115610 
  [info] Migration DoctrineMigrations\Version20210401115610 migrated (took 0.2ms, used 22M memory)
  [info] ++ migrating Bundle\DoctrineMigrations\Version20200110165135 
  [info] Migration Bundle\DoctrineMigrations\Version20200110165135 migrated (took 0.2ms, used 22M memory)
  [info] ++ migrating Bundle\DoctrineMigrations\Version20210319153423
  [info] Migration Bundle\DoctrineMigrations\Version20210319153423 migrated (took 0.3ms, used 22M memory)
  [info] ++ migrating Bundle\DoctrineMigrations\Version20210319163957 
  [info] Migration Bundle\DoctrineMigrations\Version20210319163957 migrated (took 0.1ms, used 22M memory)
  [info] ++ migrating Bundle\DoctrineMigrations\Version20210330132940 
  [info] Migration Bundle\DoctrineMigrations\Version20210330132940 migrated (took 0.2ms, used 22M memory)
  [info] ++ migrating Bundle\DoctrineMigrations\Version20210331160220 
  [info] Migration Bundle\DoctrineMigrations\Version20210331160220 migrated (took 0.2ms, used 22M memory)
  [notice] finished in 52.7ms, used 22M memory, 9 migrations executed, 36 sql queries

Expected behavior

Migrations are executed in Order the config defines

[pounard]

Upon the migrate command the migrations are first sorted by namespace and afterwards by year and month

I had the same problem, but in general, not only because of the organize_migrations setting.

Fact is, default ordering is simply using migration names (if I remember correctly). For changing this, please see the https://github.com/doctrine/migrations/issues/1074 issue, more specifically @goetas comment: https://github.com/doctrine/migrations/issues/1074#issuecomment-761556707 which links to to a tutorial on how to write a custom VersionComparator for ordering migrations.

I had the exact same issue, we run various app versions at the same time (dev, qa, preprod and prod) and when we merge altogether branches we end up with migration not respecting the version and date order, so I did that (this is Symfony code, but may be adapted easily):

services:
    App\Infrastructure\Doctrine\Migrations\VersionComparator:
        autowire: true
doctrine_migrations:
    migrations_paths:
        # ...
        Migration\_0_9_0: "%kernel.project_dir%/src/Persistence/System/Migration/_0_9_0"
        Migration\_0_10_0: "%kernel.project_dir%/src/Persistence/System/Migration/_0_11_0"
        Migration\_0_11_0: "%kernel.project_dir%/src/Persistence/System/Migration/_0_11_0"
    services:
        'Doctrine\Migrations\Version\Comparator': App\Infrastructure\Doctrine\Migrations\VersionComparator

Then custom App\Infrastructure\Doctrine\Migrations\VersionComparator.php file (warninig: this is quick and dirty code because we had our QA env broken, so I had to do it fast, not right):

<?php

declare(strict_types=1);

namespace App\Infrastructure\Doctrine\Migrations;

use Doctrine\Migrations\Version\Comparator;
use Doctrine\Migrations\Version\Version;

class VersionComparator implements Comparator
{
    public function compare(Version $a, Version $b): int
    {
        list ($aDateStamp, $aAppVersion) = $this->extractComparableString($a);
        list ($bDateStamp, $bAppVersion) = $this->extractComparableString($b);

        if ($aAppVersion && $bAppVersion) {
            if ($aAppVersion === $bAppVersion) {
                return $aDateStamp <=> $bDateStamp;
            } else {
                return \version_compare($aAppVersion, $bAppVersion);
            }
        } else {
            return $aDateStamp <=> $bDateStamp;
        }
    }

    private function extractComparableString(Version $version): array
    {
        $pieces = \explode('\\', (string) $version);
        $dateStamp = null;
        $appVersion = null;

        // Default is to use time instead.
        foreach ($pieces as $piece) {
            if (\preg_match('/^Version\d+$/', $piece)) {
                $dateStamp = \substr($piece, 7);
            }
        }

        if (!$dateStamp) {
            // Now is good enough, it's impossible in theory to have
            // migrations in the future.
            $dateStamp = \date('YmdHis');
        }

        // Exemple: Migration\\_0_6_0\\Version20210312133201
        if (isset($pieces[1]) && \preg_match('/^(_|)\d+_\d+(|_\d+)$/', $pieces[1])) {
            $appVersion = \trim(\str_replace('_', '.', $pieces[1]), ".");
        }

        return [$dateStamp, $appVersion];
    }
}

And that's it (the original tutorial is not clear about how to implement it correctly).

This is an example, you'd have to deal with your namespaces in another way of course, but it gives you a good place to start.

But, in my opinion, doctrine migrations should provide per default a comparator that uses datestamps to order, not migration names. I think this limitation probably has caused many people quite some trouble.

[NeilMasters]

@pounard thanks for providing that - put me on the right track.

[beregovoy]

May be fixed https://github.com/doctrine/migrations/pull/1421