Django thinks migrations are not applied

[unformatt]

trafficstars

Not sure if this is related to django-pyodbc-azure (but I think it is because I tested with MySQL also). So first I run migrations:

>>> python manage.py migrate                                                                                                                                               
Operations to perform:
  Apply all migrations: admin, contenttypes, auth, sessions
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying sessions.0001_initial... OK

Then try to run django, it warns me about migrations:

>>> ./manage.py runserver                                                                                                                                                  
Performing system checks...

System check identified no issues (0 silenced).

You have unapplied migrations; your app may not work properly until they are applied.
Run 'python manage.py migrate' to apply them.

So try to run migrations again (which fails because I already did):

python manage.py migrate                                                                                                                                                (master)
Operations to perform:
  Apply all migrations: admin, contenttypes, auth, sessions
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "env/lib/python2.7/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "env/lib/python2.7/site-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "env/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "env/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "env/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 200, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 92, in migrate
    self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 198, in apply_migration
    state = migration.apply(state, schema_editor)
  File "env/lib/python2.7/site-packages/django/db/migrations/migration.py", line 123, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "env/lib/python2.7/site-packages/django/db/migrations/operations/models.py", line 59, in database_forwards
    schema_editor.create_model(model)
  File "env/lib/python2.7/site-packages/sql_server/pyodbc/schema.py", line 489, in create_model
    self.execute(sql, params or None)
  File "env/lib/python2.7/site-packages/sql_server/pyodbc/schema.py", line 538, in execute
    cursor.execute(sql, params)
  File "env/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "env/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "env/lib/python2.7/site-packages/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "env/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "env/lib/python2.7/site-packages/sql_server/pyodbc/base.py", line 537, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: ('42S01', "[42S01] [FreeTDS][SQL Server]There is already an object named 'django_content_type' in the database. (2714) (SQLExecDirectW)")

[unformatt]

I've found that migrations names are getting truncated in django_migrations tables:

[shaib]

It looks like all your names get cut in half, which I'm pretty sure neither Django nor django-pyodbc-azure would do on purpose. Could there be something funny with your character encodings, which would make some component along the way use the number of characters as length in bytes (but only for the "name" column)? Perhaps a weird encoding in the file system (because that's where the migration names are taken from)?

It would probably help if you said what operating system your client is on (Mac? Linux? Cygwin?)

[unformatt]

I'm on OSX. Having dealt with MS SQL <--> python character encoding nightmares before, I agree it probably lies within there.

[unformatt]

Sorry this doesn't pertain exactly to django-pyodbc-azure but I can't find any answers. I've snooped into django/db/migrations/recorder.py and when it saves the name of the migration, it's set to u'0001_initial', when I load the same record and print it, it's '0\x00\U0009005f\x00ions\x00\uffff@\U00012068'.

I can't figure out where to set the character encoding. I've tried in django settings and odbc.ini.

I've tested this manually and the cause is definitely unicode

from django.db.migrations.recorder import MigrationRecorder
m = MigrationRecorder.Migration.objects.get(id=1)
m.name = 'the_correct_name'  # this saves correctly
m.name = u'the_correct_name'  # this doesn't save correctly
m.save()

[unformatt]

My MS SQL DB shows DEFAULT_CHARACTER_SET_NAME as "iso_1". I added 'driver_charset': 'iso-8859-1' to database OPTIONS and this has seemed to fix the issue.

@michiya - what DB charset do you use where you aren't required to have driver_charset defined?

[michiya]

Hi,

I always use UTF-8 encoding when I work with FreeTDS by setting client charset to UTF-8 in my freetds.conf. http://www.freetds.org/userguide/freetdsconf.htm

Hope this helps.