Manual test fails but runtests.py passes

[jcorbettfrank]

Hi, first I wanted to say thanks for making this library. It will be very helpful to me.

I've been getting an error when trying to work with nested repeated Protobuf messages. Whenever I migrate I get "django.db.utils.OperationalError: duplicate column name: <repeated_field_name>_index". In my attempts to debug this, I worked with the test code provided in django-pb-model/pb_model/tests. When I run django-pb-mode/runtests.py everything passes. However, if I create a separate Django project/app and copy over the models.py, tests.py, and models_pb2.py files from django-pb-model/pb_model/tests I get the same error "django.db.utils.OperationalError: duplicate column name: map_string_to_message_field_index".

Below are the steps I followed to reliably produce this error with the supplied test code.

Steps to Reproduce (Ubuntu 18.04):

1. Install Django $ pip install Django
2. Install django-pb-model $ pip install django-pb-model
3. Make an overall test directory $ mkdir test
4. Clone django-pb-model
5. In your test directory, create a Django test project and app $ cd test $ django-admin startproject pb_test $ cd pb_test $ python manage.py startapp pb_app_test
6. Delete the automatically created models.py and tests.py in the pb_app_test directory $ rm pb_app_test/models.py pb_app_test/tests.py
7. Copy the test models.py, tests.py, and models_pb2.py from the cloned django-pb-model/pb_model/tests directory into your test app $ cd test/django-pb-model/pb_model/tests $ cp tests.py models.py models_pb2.py test/pb_test/pb_app_test/
8. Edit test/pb_test/pb_test/settings.py to include pb_model and pb_app_test apps INSTALLED_APPS = [ 'pb_model', 'pb_app_test.apps.PbAppTestConfig', ..., ]
9. Make migrations in your test app (from within test/pb_test) $ python manage.py makemigrations
   • Create model Embedded
   • Create model ListWrapper
   • Create model M2MRelation
   • etc.
10. Now I can get the reproducible error from either running $ python manage.py migrate or $ python manage.py test

I've put the stack trace at the bottom. So essentially what I did was manually create a fully fledged Django project/app with the test models from django-pb-models/pb_model/tests and get an error that I do not get when I just run runtests.py. This is the same error that I've been seeing when I initially tried to use django-pb-model with my custom protobuf messages.

Creating test database for alias 'default'... Traceback (most recent call last): File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 394, in execute return Database.Cursor.execute(self, query) sqlite3.OperationalError: duplicate column name: map_string_to_message_field_index

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "manage.py", line 22, in execute_from_command_line(sys.argv) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/init.py", line 401, in execute_from_command_line utility.execute() File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/init.py", line 395, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv super().run_from_argv(argv) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv self.execute(*args, **cmd_options) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute output = self.handle(*args, **options) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/commands/test.py", line 53, in handle failures = test_runner.run_tests(test_labels) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/test/runner.py", line 684, in run_tests old_config = self.setup_databases(aliases=databases) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/test/runner.py", line 606, in setup_databases self.parallel, **kwargs File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/test/utils.py", line 173, in setup_databases serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE', True), File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/base/creation.py", line 72, in create_test_db run_syncdb=True, File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/init.py", line 168, in call_command return command.execute(*args, **defaults) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute output = self.handle(*args, **options) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped res = handle_func(*args, **kwargs) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 233, in handle fake_initial=fake_initial, File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration state = migration.apply(state, schema_editor) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply operation.database_forwards(self.app_label, schema_editor, old_state, project_state) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/operations/models.py", line 92, in database_forwards schema_editor.create_model(model) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 324, in create_model self.execute(sql, params or None) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 142, in execute cursor.execute(sql, params) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 68, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers return executor(sql, params, many, context) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 86, in _execute return self.cursor.execute(sql, params) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/utils.py", line 90, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql) File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 394, in execute return Database.Cursor.execute(self, query) django.db.utils.OperationalError: duplicate column name: map_string_to_message_field_index