self-hosted icon indicating copy to clipboard operation
self-hosted copied to clipboard

Migrations not working in 24.6.0 -> DuplicateObject('constraint "sentry_externalactor_actor_id_a8478274_fk_sentry_actor_id" for relation "sentry_externalactor" already exists\n')

Open thedevale opened this issue 1 year ago • 7 comments

Self-Hosted Version

24.6.0

CPU Architecture

x86_64

Docker Version

Docker version 26.1.4, build 5650f9b

Docker Compose Version

Docker Compose version v2.27.1

Steps to Reproduce

Take a normal 23.6.2 version, fully working, and try to upgrade to 24.6.0. When running migrations, the system crashes

Expected Result

System building correctly

Actual Result

DuplicateObject('constraint "sentry_externalactor_actor_id_a8478274_fk_sentry_actor_id" for relation "sentry_externalactor" already exists\n')

`Running migrations for default Operations to perform: Apply all migrations: auth, contenttypes, feedback, hybridcloud, nodestore, replays, sentry, ses Running migrations: Applying sentry.0551_drop_xactor_actor...Traceback (most recent call last): File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner return func(self, sql, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 85, in execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ psycopg2.errors.DuplicateObject: constraint "sentry_externalactor_actor_id_a8478274_fk_sentry_acto

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner raise_the_exception(self.db, e) File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.traceback) File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner return func(self, sql, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 85, in execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ psycopg2.errors.DuplicateObject: DuplicateObject('constraint "sentry_externalactor_actor_id_a84782 SQL: ALTER TABLE "sentry_externalactor" ADD CONSTRAINT "sentry_externalactor_actor_id_a8478274_fk

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/.venv/bin/sentry", line 4, in raise SystemExit(main()) ^^^^^^ File "/usr/src/sentry/src/sentry/runner/main.py", line 148, in main func(**kwargs) File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in call return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/runner/decorators.py", line 35, in inner return ctx.invoke(f, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/runner/commands/upgrade.py", line 148, in upgrade _upgrade( File "/usr/src/sentry/src/sentry/runner/commands/upgrade.py", line 64, in _upgrade dj_call_command( File "/.venv/lib/python3.11/site-packages/django/core/management/init.py", line 194, in call return command.execute(*args, **defaults) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute output = self.handle(*args, **options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper res = handle_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 356, post_migrate_state = executor.migrate( ^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrat state = self._migrate_all_forwards( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in migra state = self.apply_migration( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/new_migrations/monkey/executor.py", line 141, in apply_migratio return super().apply_migration(state, migration, fake=fake, fake_initial=fake_initial) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply state = migration.apply(state, schema_editor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/new_migrations/migrations.py", line 22, in apply return super().apply(project_state, schema_editor, collect_sql) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply operation.database_forwards( File "/.venv/lib/python3.11/site-packages/django/db/migrations/operations/fields.py", line 235, schema_editor.alter_field(from_model, from_field, to_field) File "/usr/src/sentry/src/sentry/db/postgres/schema.py", line 52, in inner func(self, *args, **kwargs) File "/.venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 893, in alter self._alter_field( File "/.venv/lib/python3.11/site-packages/django/db/backends/postgresql/schema.py", line 288, in super()._alter_field( File "/.venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 1254, in _alt self.execute( File "/.venv/lib/python3.11/site-packages/django_zero_downtime_migrations/backends/postgres/sche super().execute(statement, params) File "/.venv/lib/python3.11/site-packages/django/db/backends/postgresql/schema.py", line 48, in return super().execute(sql, None) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 202, in execu cursor.execute(sql, params) File "/.venv/lib/python3.11/site-packages/sentry_sdk/utils.py", line 1711, in runner return sentry_patched_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/sentry_sdk/integrations/django/init.py", line 629, result = real_execute(self, sql, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute return self._execute_with_wrappers( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_wit return executor(sql, params, many, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute with self.db.wrap_database_errors: File "/.venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner raise_the_exception(self.db, e) File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.traceback) File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner return func(self, sql, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 85, in execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ django.db.utils.ProgrammingError: DuplicateObject('constraint "sentry_externalactor_actor_id_a8478 SQL: ALTER TABLE "sentry_externalactor" ADD CONSTRAINT "sentry_externalactor_actor_id_a8478274_fk Error in install/set-up-and-migrate-database.sh:30. '$dcr web upgrade --create-kafka-topics' exited with status 1 -> ./install.sh:main:38 --> install/set-up-and-migrate-database.sh:source:30

`

Event ID

No response

thedevale avatar Jun 20 '24 13:06 thedevale

Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Usage: sentry upgrade [OPTIONS]
Try 'sentry upgrade --help' for help.

Error: No such option: --create-kafka-topics
Error in install/set-up-and-migrate-database.sh:30.
'$dcr web upgrade --create-kafka-topics' exited with status 2
-> ./install.sh:main:38
--> install/set-up-and-migrate-database.sh:source:30

Cleaning up...

msschl avatar Jun 21 '24 09:06 msschl

Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Usage: sentry upgrade [OPTIONS]
Try 'sentry upgrade --help' for help.

Error: No such option: --create-kafka-topics
Error in install/set-up-and-migrate-database.sh:30.
'$dcr web upgrade --create-kafka-topics' exited with status 2
-> ./install.sh:main:38
--> install/set-up-and-migrate-database.sh:source:30

Cleaning up...

I forgot to update the versions in my .env.custom file to 24.6.0

msschl avatar Jun 21 '24 09:06 msschl

It seems like it is attempting to run the migrations twice. What if you use ./sentry-admin.sh django migrate as a standalone command, then try again?

azaslavsky avatar Jun 21 '24 23:06 azaslavsky

I'm in the process of updating an instance that has been updated last in august 2023. I had to run:

ALTER TABLE sentry_externalactor
DROP CONSTRAINT sentry_externalactor_actor_id_a8478274_fk_sentry_actor_id;

xvilo avatar Jun 24 '24 10:06 xvilo

I think I find myself in a similar situation. I followed the hard-stop versions here when upgrading – but didn't upgrade docker or docker-compose at any point during this. I think this then caused some issues when I later did upgrade docker-compose to 2+ and saw that the migrations only ran after this (By then I was already on the latest sentry version) – so think I missed some migration steps.

graeme44 avatar Jun 26 '24 15:06 graeme44

We did bump the docker compose version in the last year or so. @thedevale were you able to figure out a solution here? Perhaps it may be helpful to upgrade to an intermediate version of Sentry before going to latest...we may have missed documenting some sort of hard stop in the upgrade process.

hubertdeng123 avatar Jun 27 '24 21:06 hubertdeng123

I had similar issues, got a

Usage: sentry upgrade [OPTIONS]
Try 'sentry upgrade --help' for help.

Error: No such option: --create-kafka-topics
Error in install/set-up-and-migrate-database.sh:30.
'$dcr web upgrade --create-kafka-topics' exited with status 2
-> ./install.sh:main:38
--> install/set-up-and-migrate-database.sh:source:30

when running ./install.sh.

Problem was that copying 24.6 dir contents over my current 23.12 dir, did not copy .env file (cp -r ../24.6/* .)

After copying that file over, the install.sh did complete succesfully.

peterdk avatar Jun 29 '24 23:06 peterdk

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

getsantry[bot] avatar Aug 12 '25 07:08 getsantry[bot]