self-hosted
self-hosted copied to clipboard
Published
20 hours ago •
getsentry
getsentry
Migration 0878_backfill_open_periods failed upgrading to 25.5.1
Self-Hosted Version
25.5.1
CPU Architecture
x86_64
Docker Version
25.0.8, build 0bab007
Docker Compose Version
2.37.1
Machine Specification
- [x] My system meets the minimum system requirements of Sentry
Steps to Reproduce
We were executing the upgrade path and hit an error with 0878_backfill_open_periods.
Expected Result
The migration 0878_backfill_open_periods should apply cleanly.
Actual Result
Applying sentry.0877_integer_drift_release... OK
Applying sentry.0878_backfill_open_periods...Grouped Messages
16:41:10 [INFO] sentry.migrations.0878_backfill_open_periods: Processing batch for group open period backfill (last_group_id=7767)
16:41:23 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=31 starting_activity=85099)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=135100)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=61285)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=61284)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=61283)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=61282)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=53688)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=53674)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=45942)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=45941)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=39756)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=38490)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=36721)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=29006)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=27056)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=2903 starting_activity=26887)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=5630 starting_activity=107053)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=6217 starting_activity=67349)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=6313 starting_activity=63628)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=6313 starting_activity=62509)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=6760 starting_activity=107060)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7326 starting_activity=103863)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7326 starting_activity=103683)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7326 starting_activity=93473)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7326 starting_activity=88129)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7326 starting_activity=80894)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7326 starting_activity=75814)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7394 starting_activity=106928)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7496 starting_activity=107249)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7496 starting_activity=102533)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7496 starting_activity=97107)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7496 starting_activity=96170)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7496 starting_activity=93345)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7496 starting_activity=90511)
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7676 starting_activity=96612)
Traceback (most recent call last):
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
return self.cursor.execute(sql, clean_bad_params(params))
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.ExclusionViolation: conflicting key value violates exclusion constraint "exclude_overlapping_start_end"
DETAIL: Key (group_id, tstzrange(date_started, date_ended, '[)'::text))=(2503, ["2022-11-24 11:56:56.411+00",)) conflicts with existing key (group_id, tstzrange(date_started, date_ended, '[)'::text))=(2503, ["2024-02-28 09:18:50.706556+00","2024-03-21 14:40:00.232373+00")).
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner
raise_the_exception(self.db, e)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner
return func(self, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner
return func(self, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner
raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
return self.cursor.execute(sql, clean_bad_params(params))
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.ExclusionViolation: ExclusionViolation('conflicting key value violates exclusion constraint "exclude_overlapping_start_end"\nDETAIL: Key (group_id, tstzrange(date_started, date_ended, \'[)\'::text))=(2503, ["2022-11-24 11:56:56.411+00",)) conflicts with existing key (group_id, tstzrange(date_started, date_ended, \'[)\'::text))=(2503, ["2024-02-28 09:18:50.706556+00","2024-03-21 14:40:00.232373+00")).\n')
SQL: INSERT INTO "sentry_groupopenperiod" ("date_updated", "date_added", "user_id", "date_started", "date_ended", "data", "group_id", "project_id", "resolution_activity_id") VALUES ...truncated..., (%s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "sentry_groupopenperiod"."id"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/src/sentry/src/sentry/migrations/0878_backfill_open_periods.py", line 174, in _backfill_group_open_periods
GroupOpenPeriod.objects.bulk_create(open_periods)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/models/query.py", line 823, in bulk_create
returned_columns = self._batched_insert(
objs_without_pk,
...<4 lines>...
unique_fields=unique_fields,
)
File "/.venv/lib/python3.13/site-packages/django/db/models/query.py", line 1892, in _batched_insert
self._insert(
~~~~~~~~~~~~^
item,
^^^^^
...<5 lines>...
returning_fields=self.model._meta.db_returning_fields,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/.venv/lib/python3.13/site-packages/django/db/models/query.py", line 1864, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py", line 1882, in execute_sql
cursor.execute(sql, params)
~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/sentry_sdk/utils.py", line 1811, in runner
return sentry_patched_function(*args, **kwargs)
File "/.venv/lib/python3.13/site-packages/sentry_sdk/integrations/django/__init__.py", line 651, in execute
result = real_execute(self, sql, params)
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
~~~~~~~~~~~~~~~~~~~~~~~~~~~^
sql, params, many=False, executor=self._execute
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner
raise_the_exception(self.db, e)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner
return func(self, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner
return func(self, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner
raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
return self.cursor.execute(sql, clean_bad_params(params))
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.IntegrityError: ExclusionViolation('conflicting key value violates exclusion constraint "exclude_overlapping_start_end"\nDETAIL: Key (group_id, tstzrange(date_started, date_ended, \'[)\'::text))=(2503, ["2022-11-24 11:56:56.411+00",)) conflicts with existing key (group_id, tstzrange(date_started, date_ended, \'[)\'::text))=(2503, ["2024-02-28 09:18:50.706556+00","2024-03-21 14:40:00.232373+00")).\n')
SQL: INSERT INTO "sentry_groupopenperiod" ("date_updated", "date_added", "user_id", "date_started", "date_ended", "data", "group_id", "project_id", "resolution_activity_id") VALUES ...truncated..., (%s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "sentry_groupopenperiod"."id"
16:41:26 [ERROR] sentry.migrations.0878_backfill_open_periods: Error creating open period (group_ids=[31, 53, 101, 302, 365, 462, 469, 523, 556, 619, 871, 1406, 1442, 1447, 1521, 1558, 1579, 1596, 2099, 2100, 2241, 2282, 2451, 2503, 2519, 2520, 2903, 2999, 3218, 3539, 3560, 3689, 3900, 3986, 4008, 4152, 5004, 5225, 5381, 5544, 5630, 5686, 6148, 6217, 6242, 6313, 6443, 6697, 6698, 6730, 6760, 6784, 6797, 6815, 6863, 6870, 6920, 6957, 6969, 7014, 7173, 7180, 7186, 7187, 7188, 7192, 7193, 7202, 7205, 7226, 7227, 7246, 7264, 7297, 7326, 7359, 7360, 7394, 7404, 7408, 7430, 7463, 7466, 7467, 7488, 7496, 7505, 7510, 7554, 7577, 7592, 7635, 7649, 7672, 7676, 7691, 7693, 7734, 7763, 7767] error=IntegrityError('ExclusionViolation(\'conflicting key value violates exclusion constraint "exclude_overlapping_start_end"\\nDETAIL: Key (group_id, tstzrange(date_started, date_ended, \\\'[)\\\'::text))=(2503, ["2022-11-24 11:56:56.411+00",)) conflicts with existing key (group_id, tstzrange(date_started, date_ended, \\\'[)\\\'::text))=(2503, ["2024-02-28 09:18:50.706556+00","2024-03-21 14:40:00.232373+00")).\\n\')\nSQL: INSERT INTO "sentry_groupopenperiod" ("date_updated", "date_added", "user_id", "date_started", "date_ended", "data", "group_id", "project_id", "resolution_activity_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s), ...truncated for brevity..., (%s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "sentry_groupopenperiod"."id"'))
16:41:27 [INFO] sentry.migrations.0878_backfill_open_periods: Processing batch for group open period backfill (last_group_id=8288)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7982 starting_activity=92183)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7982 starting_activity=92182)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7982 starting_activity=86875)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=7982 starting_activity=86874)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=8046 starting_activity=136914)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=8046 starting_activity=136913)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=8046 starting_activity=133130)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=8046 starting_activity=123754)
16:41:29 [ERROR] sentry.migrations.0878_backfill_open_periods: No end activity found for group open period backfill (group_id=8046 starting_activity=110589)
Traceback (most recent call last):
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
return self.cursor.execute(sql, clean_bad_params(params))
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.DataException: range lower bound must be less than or equal to range upper bound
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner
raise_the_exception(self.db, e)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner
return func(self, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner
return func(self, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner
raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
return self.cursor.execute(sql, clean_bad_params(params))
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.DataException: DataException('range lower bound must be less than or equal to range upper bound\n')
SQL: INSERT INTO "sentry_groupopenperiod" ("date_updated", "date_added", "user_id", "date_started", "date_ended", "data", "group_id", "project_id", "resolution_activity_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s), ....truncated for brevity..., (%s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "sentry_groupopenperiod"."id"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/.venv/bin/sentry", line 4, in <module>
raise SystemExit(main())
~~~~^^
File "/usr/src/sentry/src/sentry/runner/main.py", line 145, in main
func(**kwargs)
~~~~^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/.venv/lib/python3.13/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/src/sentry/src/sentry/runner/decorators.py", line 34, in inner
return ctx.invoke(f, *args, **kwargs)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/src/sentry/src/sentry/runner/commands/upgrade.py", line 157, in upgrade
_upgrade(
~~~~~~~~^
not noinput,
^^^^^^^^^^^^
...<5 lines>...
create_kafka_topics,
^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/src/sentry/src/sentry/runner/commands/upgrade.py", line 73, in _upgrade
dj_call_command(
~~~~~~~~~~~~~~~^
"migrate",
^^^^^^^^^^
...<3 lines>...
verbosity=verbosity,
^^^^^^^^^^^^^^^^^^^^
)
^
File "/.venv/lib/python3.13/site-packages/django/core/management/__init__.py", line 194, in call_command
return command.execute(*args, **defaults)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/core/management/base.py", line 460, in execute
output = self.handle(*args, **options)
File "/.venv/lib/python3.13/site-packages/django/core/management/base.py", line 107, in wrapper
res = handle_func(*args, **kwargs)
File "/.venv/lib/python3.13/site-packages/django/core/management/commands/migrate.py", line 353, in handle
post_migrate_state = executor.migrate(
targets,
...<3 lines>...
fake_initial=fake_initial,
)
File "/.venv/lib/python3.13/site-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
state, plan, full_plan, fake=fake, fake_initial=fake_initial
)
File "/.venv/lib/python3.13/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
state, migration, fake=fake, fake_initial=fake_initial
)
File "/usr/src/sentry/src/sentry/new_migrations/monkey/executor.py", line 154, in apply_migration
return super().apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/migrations/executor.py", line 255, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/src/sentry/src/sentry/new_migrations/migrations.py", line 28, in apply
return super().apply(project_state, schema_editor, collect_sql)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/migrations/migration.py", line 132, in apply
operation.database_forwards(
~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self.app_label, schema_editor, old_state, project_state
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/.venv/lib/python3.13/site-packages/django/db/migrations/operations/special.py", line 196, in database_forwards
self.code(from_state.apps, schema_editor)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/migrations/0878_backfill_open_periods.py", line 202, in backfill_group_open_periods
_backfill_group_open_periods(apps, group_data)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/migrations/0878_backfill_open_periods.py", line 174, in _backfill_group_open_periods
GroupOpenPeriod.objects.bulk_create(open_periods)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/models/query.py", line 823, in bulk_create
returned_columns = self._batched_insert(
objs_without_pk,
...<4 lines>...
unique_fields=unique_fields,
)
File "/.venv/lib/python3.13/site-packages/django/db/models/query.py", line 1892, in _batched_insert
self._insert(
~~~~~~~~~~~~^
item,
^^^^^
...<5 lines>...
returning_fields=self.model._meta.db_returning_fields,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/.venv/lib/python3.13/site-packages/django/db/models/query.py", line 1864, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py", line 1882, in execute_sql
cursor.execute(sql, params)
~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/sentry_sdk/utils.py", line 1811, in runner
return sentry_patched_function(*args, **kwargs)
File "/.venv/lib/python3.13/site-packages/sentry_sdk/integrations/django/__init__.py", line 651, in execute
result = real_execute(self, sql, params)
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
~~~~~~~~~~~~~~~~~~~~~~~~~~~^
sql, params, many=False, executor=self._execute
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/.venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner
raise_the_exception(self.db, e)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner
return func(self, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner
return func(self, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner
raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
return self.cursor.execute(sql, clean_bad_params(params))
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.DataError: DataException('range lower bound must be less than or equal to range upper bound\n')
SQL: INSERT INTO "sentry_groupopenperiod" ("date_updated", "date_added", "user_id", "date_started", "date_ended", "data", "group_id", "project_id", "resolution_activity_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s), .... truncated for brevity ..., (%s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "sentry_groupopenperiod"."id"
Error in install/set-up-and-migrate-database.sh:31.
'$dcr web upgrade --create-kafka-topics' exited with status 1
-> ./install.sh:main:41
--> install/set-up-and-migrate-database.sh:source:31
Cleaning up...
Event ID
No response
We've been able to bypass the issue by:
# Logging into postgres
psql -U postgres -h localhost
And skipping the migration:
INSERT INTO django_migrations (app, name, applied) VALUES ('sentry', '0878_backfill_open_periods', NOW());
It seems to have worked ok, but I'm not sure what we will have broken as a result. 🤞 nothing major... 😬
Hello,
in https://github.com/getsentry/sentry/blob/master/src/sentry/migrations/0925_backfill_open_periods.py#L170 is already IntegrityError catched. I've added also DataError, migrations run completely, some chunks of 0878 migration failed, but probably it is not a fatal error so ignoring it is OK.
Thanks for fixing this @BYK !
What would you recommend if we've bypassed this migration (appreciate we probably shouldn't have done...)?
Is it best to unmark it as complete so it runs again, or best to leave as is? I'm not familiar enough with the data model to fully understand what it does.
Will there be a release with this fix? We're stuck on this error when upgrading to 25.5.1, which as a hard stop, prevents us from getting past this if the fix is only in newer releases.
@edhgoose
What would you recommend if we've bypassed this migration (appreciate we probably shouldn't have done...)?
I guess you might be able to run this manually? That said if you don't have any issues, I'd just apply any table modifications and call it a day (skip data migration as clearly you don't miss that?)
@WilliamNsight
Will there be a release with this fix? We're stuck on this error when upgrading to 25.5.1, which as a hard stop, prevents us from getting past this if the fix is only in newer releases.
Yeah we're planning a release. @aldy505 and @hubertdeng123 will be on it soon.
Thanks @aldy505 / @BYK - appreciate the updates. We'll try the update and report back if we have problems. Ta!
I attempted an upgrade with the 25.6.2 release, but again, I was not able to complete the upgrade because of the 25.5.1 hard stop. We are currently running 24.12.1, and cannot upgrade to 25.5.1 due to the migration error, therefore I am unable to upgrade to 25.6.2. Basically I need a 25.5.2, which contains the fix, and replaces 25.5.1 as the hard stop. Unless there's a version between 24.12.1 and 25.5.1 that may resolve the migration issue.
I attempted an upgrade with the 25.6.2 release, but again, I was not able to complete the upgrade because of the 25.5.1 hard stop. We are currently running 24.12.1, and cannot upgrade to 25.5.1 due to the migration error, therefore I am unable to upgrade to 25.6.2. Basically I need a 25.5.2, which contains the fix, and replaces 25.5.1 as the hard stop. Unless there's a version between 24.12.1 and 25.5.1 that may resolve the migration issue.
@WilliamNsight would you like to try to upgrade directly to 25.6.2, skipping 25.5.1? It's not too far away, you should be good. But to be sure, backup everything first! Safest option would be to create a snapshot of your server first.
@aldy505
@WilliamNsight would you like to try to upgrade directly to 25.6.2, skipping 25.5.1? It's not too far away, you should be good. But to be sure, backup everything first! Safest option would be to create a snapshot of your server first.
If this works, we should probably update our docs and our hard-stop. I think the need for the hard-stop is due to the migrations squashing so I'm not sure if it's gonna work.
I did try to upgrade directly to 25.6.2, skipping 25.5.1, but the installer will not finish because of the hard stop at 25.5.1. 25.5.1 fails with the migration error. I cannot continue with the current hard stop, and cannot upgrade to 25.5.1 or newer because of the migration error during the install/upgrade
@WilliamNsight What error are you getting when upgrading to 25.6.2? It will help us debug the error
Current version: Sentry 24.12.1
When I attempt to upgrade to 25.6.2 directly, I get:
# ./install.sh
...
Error opening GeoIP database: /geoip/GeoLite2-City.mmdb
Error opening GeoIP database in Rust: /geoip/GeoLite2-City.mmdb
Error: It looks like you've skipped a hard stop in our upgrade process. Please follow the upgrade process here: https://develop.sentry.dev/self-hosted/releases/#hard-stops
Error in install/set-up-and-migrate-database.sh:22.
'$dcr web upgrade --create-kafka-topics' exited with status 1
-> ./install.sh:main:42
--> install/set-up-and-migrate-database.sh:source:22
Cleaning up...
We do have a test Sentry server that we were able to upgrade from 24.12.1 -> 25.5.1 -> 25.6.2 without encountering the migration error mentioned here. However, we cannot upgrade our production server because we can't get around both the migration error and the hard stop requirement.
By patching install/set-up-and-migrate-database.sh:
diff --git a/install/set-up-and-migrate-database.sh b/install/set-up-and-migrate-database.sh
index 2d4e132..0deac66 100644
--- a/install/set-up-and-migrate-database.sh
+++ b/install/set-up-and-migrate-database.sh
@@ -28,6 +28,7 @@ with connection.cursor() as cursor:
echo " $dc_base run --rm web createuser"
echo ""
else
+ $dcr -it --entrypoint /bin/bash web
$dcr web upgrade --create-kafka-topics
fi
else
I was able to enter the container while install.sh was running. There, I simply patched src/sentry/migrations /0878_backfill_open_periods.py using the patch linked above, meant for 0925_backfill_open_periods.py:
# apt update
# apt install nano
# nano src/sentry/migrations/0878_backfill_open_periods.py
<manually add DataError to imports and try-catch>
# sentry web upgrade --create-kafka-topics
# exit
Migrations then finished "just fine", assuming we can safely ignore:
Traceback (most recent call last):
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
return self.cursor.execute(sql, clean_bad_params(params))
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.DataException: range lower bound must be less than or equal to range upper bound
Traceback (most recent call last):
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
return self.cursor.execute(sql, clean_bad_params(params))
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.ExclusionViolation: conflicting key value violates exclusion constraint "exclude_overlapping_start_end"
DETAIL: Key (group_id, tstzrange(date_started, date_ended, '[)'::text))=(27868, ["2025-06-03 15:38:41.361589+00",)) conflicts with existing key (group_id, tstzrange(date_started, date_ended, '[)'::text))=(27868, ["2025-06-04 09:37:22.366895+00",)).
I think this is what @cuchac encountered back in June, and how they ran migrations, based on their description.
The instructions provided by @kostirez1 did the trick! I was able to get past the migration issue and upgrade to v25.5.1, and ultimately v25.6.2.
The only difference from the above instructions is that the sentry command is sentry upgrade --create-kafka-topics.
Thanks!