Error during application of M2M migrations

[raphaelm]

During running an auto-generated migration, I run into

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/commands/sqlmigrate.py", line 30, in execute
    return super(Command, self).execute(*args, **options)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/commands/sqlmigrate.py", line 61, in handle
    sql_statements = executor.collect_sql(plan)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/migrations/executor.py", line 82, in collect_sql
    migration.apply(project_state, schema_editor, collect_sql=True)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/migrations/migration.py", line 108, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/migrations/operations/fields.py", line 139, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/backends/schema.py", line 445, in alter_field
    return self._alter_many_to_many(model, old_field, new_field, strict)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/backends/sqlite3/schema.py", line 229, in _alter_many_to_many
    old_field.rel.through._meta.get_field_by_name(old_field.m2m_reverse_field_name())[0],
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/utils/functional.py", line 17, in _curried
    return _curried_func(*(args + moreargs), **dict(kwargs, **morekwargs))
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/models/fields/related.py", line 2221, in _get_m2m_reverse_attr
    return getattr(self, cache_attr)
AttributeError: 'VersionedManyToManyField' object has no attribute '_m2m_reverse_name_cache'

The migration looks like

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
import versions.models
import pretixbase.models


class Migration(migrations.Migration):

    dependencies = [
        ('pretixbase', '0002_auto_20150211_2031'),
    ]

    operations = [
        migrations.AlterField(
            model_name='quota',
            name='items',
            field=versions.models.VersionedManyToManyField(verbose_name='Item', to='pretixbase.Item', blank=True, related_name='quotas'),
            preserve_default=True,
        ),
    ]

I'm not 100% sure but I'm fairly certain that the migration was created because I explicitely set the related_name attribute, while it was auto-generated before. Do you think there is an easy way to solve this?

[brandonmoser]

I'm receiving the same error while trying to run a migration with an VM2M field, but was unable to get the above workaround to fix the error. Here is the snippet of the migration that references a VersionedManyToManyField.

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
import datetime
from django.utils.timezone import utc
import versions.models
import model_utils.fields
import shared.utils
import django_hstore.fields
import django_hstore.virtual
import ckeditor.fields
import django.utils.timezone


class Migration(migrations.Migration):

    dependencies = [
        ('django_hstore', '__first__'),
        ('webassess', '0002_remove_assessmentquestions'),
    ]

    operations = [
        ...
        migrations.AlterField(
            model_name='assessmentbattery',
            name='assessments',
            field=versions.models.VersionedManyToManyField(to='webassess.Assessment'),
            preserve_default=True,
        ),
        ...
    ]

If I add a related_name= attribute, as suggested above, it still fails with the same error, _AttributeError: 'VersionedManyToManyField' object has no attribute 'm2m_reverse_name_cache'

Here is my traceback for reference (similar to above TB, except Python2.7):

Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/sqlmigrate.py", line 30, in execute
    return super(Command, self).execute(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/sqlmigrate.py", line 61, in handle
    sql_statements = executor.collect_sql(plan)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 82, in collect_sql
    migration.apply(project_state, schema_editor, collect_sql=True)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 108, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 139, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 460, in alter_field
    return self._alter_many_to_many(model, old_field, new_field, strict)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 764, in _alter_many_to_many
    new_field.rel.through._meta.get_field_by_name(new_field.m2m_reverse_field_name())[0],
  File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 17, in _curried
    return _curried_func(*(args + moreargs), **dict(kwargs, **morekwargs))
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/related.py", line 2224, in _get_m2m_reverse_attr
    return getattr(self, cache_attr)
AttributeError: 'VersionedManyToManyField' object has no attribute '_m2m_reverse_name_cache'

[maennel]

So, I've managed to write a test case that reproduces this issue reliably (see branch bugfix/reproduce_failing_vm2m_migration and the commit above). Unfortunately, I'll have to postpone the follow-up to some other day...

[maennel]

So far, I've managed to work around the first error that is described above. However, there's more! ;) It seems that in general, Django migrations don't get along very well with M2M-intermediary models that are Versionable at the same time (so, basically any VersionedManyToManyField being defined). Finally I got to the following error:

Error
Traceback (most recent call last):
  File "/vagrant/migrations_tests/tests/test_migrations.py", line 55, in test_migration
    self.assertRaises(AttributeError, self.run_migration)
  File "/usr/lib/python2.7/unittest/case.py", line 475, in assertRaises
    callableObj(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django_migration_testcase/django_migrations.py", line 39, in run_migration
    verbosity=0, no_initial_data=True)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 115, in call_command
    return klass.execute(*args, **defaults)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 161, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 68, in migrate
    self.apply_migration(migration, fake=fake)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 102, in apply_migration
    migration.apply(project_state, schema_editor)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 108, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 139, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 460, in alter_field
    return self._alter_many_to_many(model, old_field, new_field, strict)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/schema.py", line 234, in _alter_many_to_many
    override_uniques=(new_field.m2m_field_name(), new_field.m2m_reverse_field_name()),
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/schema.py", line 135, in _remake_table
    temp_model = type(model._meta.object_name, model.__bases__, body)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 229, in __new__
    'base class %r' % (field.name, name, base.__name__)
FieldError: Local field 'id' in class 'mymigratingmodelb_a_models' clashes with field of similar name from base class 'Versionable'

It's raised because during a migration, new intermediary classes are created, and, since deepcopied from an already existing intermediary class, are conflicting when their parent class' (Versionable) properties are being set up (id, identity, version_from, etc.). I've committed my stuff to the following branch: https://github.com/swisscom/cleanerversion/tree/bugfix/fix_failing_vm2m_migration I'm doing tests on Django 1.7.10

[maennel]

I've been pointed to Django's ticket #25154 which was obviously opened by @brandonmoser. Thanks for that, I think the way it's going will be helpful. Also, there's a patch suggested by @MarkusH which can be found here which could help for further debugging.