self-hosted
self-hosted copied to clipboard
Published
20 hours ago •
getsentry
getsentry
IntegrityError UniqueViolation when restoring a backup. I'd like to restore a partial backup onto a completely clean installation of Sentry but I keep getting a duplicate key value error when restoring the backup..
Self-Hosted Version
24.6.0
CPU Architecture
x86_64
Docker Version
27.0.3
Docker Compose Version
2.28.1
Steps to Reproduce
These are the steps I took:
- Updated Sentry to version 24.6.0
- Created a partial backup using
./scripts/backup.sh global - Completely reset Docker and deleted any remaining Docker directories/files
- Reinstalled Docker
- Ran
./install.shfrom my Sentry directory (24.6.0)
At this point I tried running the restore but encountered an error.
I then tried running docker compose up -d and let it finish and tried running the restore but encountered another error.
I then tried logging in to Sentry and then running the restore but it still fails.
The documentation site states:
We strongly recommended that you restore your backup on the same version of Sentry on a fresh install (empty database but migrations are run)
So I assume this means that the restore should be run after ./install.sh completes.
But I'm unsure as to what migrations are or when they are run. Is this at the end of the install script? Is this after I call 'docker compose up -d`? Or is this after logging into the Sentry portal?
When exactly am I supposed to trigger the restore?
Expected Result
Expected a successful restore.
Actual Result
This is the restore log I see after a COMPLETELY FRESH INSTALL. This is after ./install.sh completes and after running docker compose up -d and waiting about 5 minutes.
sentryroot@AZURE-SENTRY:~/sentry$ ./scripts/restore.sh global
▶ Initializing Docker Compose ...
▶ Detecting Docker platform
Detected Docker platform is linux/amd64
▶ Setting up error handling ...
#0 building with "default" instance using docker driver
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 292B done
#1 DONE 0.0s
#2 [internal] load metadata for docker.io/library/debian:bookworm-slim
#2 DONE 0.4s
#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s
#4 [1/2] FROM docker.io/library/debian:bookworm-slim@sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75510a9d74bb5ec535b33
#4 DONE 0.0s
#5 [2/2] RUN set -x && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends jq && apt-get clean && rm -rf /var/lib/apt/lists/*
#5 CACHED
#6 exporting to image
#6 exporting layers done
#6 writing image sha256:fa67d561b504496dc6088bdfa5d62323c1b4a335ede13d8f57728f7811b56e8b done
#6 naming to docker.io/library/sentry-self-hosted-jq-local done
#6 DONE 0.0s
latest: Pulling from getsentry/sentry-cli
Digest: sha256:a0a7d8066341778703d68a74623f30e264b6c1e42e88064a6dd08fbced42eceb
Status: Image is up to date for getsentry/sentry-cli:latest
docker.io/getsentry/sentry-cli:latest
Container sentry-self-hosted-smtp-1 Running
Container sentry-self-hosted-symbolicator-1 Running
Container sentry-self-hosted-memcached-1 Running
Container sentry-self-hosted-redis-1 Running
Container sentry-self-hosted-clickhouse-1 Running
Container sentry-self-hosted-zookeeper-1 Running
Container sentry-self-hosted-postgres-1 Running
Container sentry-self-hosted-kafka-1 Running
Container sentry-self-hosted-vroom-1 Running
Container sentry-self-hosted-snuba-outcomes-billing-consumer-1 Running
Container sentry-self-hosted-snuba-transactions-consumer-1 Running
Container sentry-self-hosted-snuba-replacer-1 Running
Container sentry-self-hosted-snuba-errors-consumer-1 Running
Container sentry-self-hosted-snuba-outcomes-consumer-1 Running
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Running
Container sentry-self-hosted-snuba-api-1 Running
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Running
Container sentry-self-hosted-zookeeper-1 Waiting
Container sentry-self-hosted-zookeeper-1 Healthy
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Proceeding with this operation will irrecoverably delete all existing
low-volume data - are you sure want to continue? [y/n]: y
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.integration' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.integration import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.option' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.option import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.organization' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.organization import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.organizationintegration' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.organizationintegration import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.organizationoption' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.organizationoption import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.relay' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.relay import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.relayusage' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.relayusage import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.repository' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.repository import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.team' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.team import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.user' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.user import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:53 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.userip' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.userip import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.userip' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=21)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.userip import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=21)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.userrole' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.userrole import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.userroleuser' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.userroleuser import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.recentsearch' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.recentsearch import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.recentsearch' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=21)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.recentsearch import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=21)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.recentsearch' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=41)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.recentsearch import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=41)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.recentsearch' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=61)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.recentsearch import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=61)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.recentsearch' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=81)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.recentsearch import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=81)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.project' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.project import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.organizationmember' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.services.hybrid_cloud.import_export.impl: import_by_model.successfully_imported (model_name=NormalizedModelName: sentry.organizationmember import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
16:01:54 [INFO] sentry.backup.imports: import_by_model.request_import (model_name='sentry.environment' import_uuid='3dfcd12abb8d474ab48232d555a64338' min_ordinal=1)
RpcImportError(
kind: IntegrityError,
on: InstanceID(model: 'sentry.environment'),
reason: UniqueViolation('duplicate key value violates unique constraint "sentry_environment_organization_id_name_95a37dc7_uniq"\nDETAIL: Key (organization_id, name)=(1, Production) already exists.\n')
SQL: INSERT INTO "sentry_environment" ("organization_id", "name", "date_added") VALUES (%s, %s, %s) RETURNING "sentry_environment"."id"
)
>> Are you restoring from a backup of the same version of Sentry?
>> Are you restoring onto a clean database?
>> If so then this IntegrityError might be our fault, you can open an issue here:
>> https://github.com/getsentry/sentry/issues/new/choose
RpcImportError(
kind: IntegrityError,
on: InstanceID(model: 'sentry.environment'),
reason: UniqueViolation('duplicate key value violates unique constraint "sentry_environment_organization_id_name_95a37dc7_uniq"\nDETAIL: Key (organization_id, name)=(1, Production) already exists.\n')
SQL: INSERT INTO "sentry_environment" ("organization_id", "name", "date_added") VALUES (%s, %s, %s) RETURNING "sentry_environment"."id"
)
Traceback (most recent call last):
File "/.venv/bin/sentry", line 4, in <module>
raise SystemExit(main())
^^^^^^
File "/usr/src/sentry/src/sentry/runner/main.py", line 148, in main
func(**kwargs)
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/runner/decorators.py", line 35, in inner
return ctx.invoke(f, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/runner/commands/backup.py", line 898, in import_global
import_in_global_scope(
File "/usr/src/sentry/src/sentry/backup/imports.py", line 598, in import_in_global_scope
return _import(
^^^^^^^^
File "/usr/src/sentry/src/sentry/backup/imports.py", line 474, in _import
do_writes(pk_map)
File "/usr/src/sentry/src/sentry/backup/imports.py", line 403, in do_writes
do_write(import_write_context, pk_map, model_name, json_data, offset)
File "/usr/src/sentry/src/sentry/backup/imports.py", line 334, in do_write
raise ImportingError(result)
sentry.backup.imports.ImportingError: is_err=True kind='IntegrityError' on=InstanceID(model='sentry.environment', ordinal=None) left_pk=None right_pk=None reason='UniqueViolation(\'duplicate key value violates unique constraint "sentry_environment_organization_id_name_95a37dc7_uniq"\\nDETAIL: Key (organization_id, name)=(1, Production) already exists.\\n\')\nSQL: INSERT INTO "sentry_environment" ("organization_id", "name", "date_added") VALUES (%s, %s, %s) RETURNING "sentry_environment"."id"'
cat: sentry_restore: No such file or directory
cat: global_log-2024-07-15_16-01-41.txt: No such file or directory
Event ID
No response
@hubertdeng123 when doing research on this issue i noticed that you had done some work on UniqueViolation issues when restoring a backup, could you let me know if this is now fixed?
Connected to the postgres database to check what rows are conflicting in sentry_environment... but it's empty. So it's complaining about a UniqueViolation even though the table is empty?
I've attempted removing all the sentry.environment module nodes in the backup.json file, i still get a UniqueViolation duplicat key value for sentry_environment
Additionally, every time i re-run the import tool the organization_id keeps incrementing by 1.
reason: UniqueViolation(
'duplicate key value violates unique constraint "sentry_environment_organization_id_name_95a37dc7_uniq"\n
DETAIL: Key (organization_id, name)=(4, Production) already exists.\n')
SQL: INSERT INTO "sentry_environment" ("organization_id", "name", "date_added")
VALUES (%s, %s, %s) RETURNING "sentry_environment"."id"
)
Restore log:
sentryroot@CC-AZURE-SENTRY:~/sentry$ ./sentry-admin.sh import organizations backup.json
▶ Initializing Docker Compose ...
▶ Detecting Docker platform
Detected Docker platform is linux/amd64
Container sentry-self-hosted-redis-1 Running
Container sentry-self-hosted-smtp-1 Running
Container sentry-self-hosted-postgres-1 Running
Container sentry-self-hosted-memcached-1 Running
Container sentry-self-hosted-zookeeper-1 Running
Container sentry-self-hosted-symbolicator-1 Running
Container sentry-self-hosted-clickhouse-1 Running
Container sentry-self-hosted-kafka-1 Running
Container sentry-self-hosted-snuba-errors-consumer-1 Running
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Running
Container sentry-self-hosted-snuba-transactions-consumer-1 Running
Container sentry-self-hosted-snuba-outcomes-billing-consumer-1 Running
Container sentry-self-hosted-snuba-outcomes-consumer-1 Running
Container sentry-self-hosted-snuba-api-1 Running
Container sentry-self-hosted-vroom-1 Running
Container sentry-self-hosted-snuba-replacer-1 Running
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Running
Container sentry-self-hosted-zookeeper-1 Waiting
Container sentry-self-hosted-zookeeper-1 Healthy
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Still importing: 5.00 seconds elapsed.
RpcImportError(
kind: IntegrityError,
on: InstanceID(model: 'sentry.environment'),
reason: UniqueViolation('duplicate key value violates unique constraint "sentry_environment_organization_id_name_95a37dc7_uniq"\nDETAIL: Key (organization_id, name)=(4, Production) already exists.\n')
SQL: INSERT INTO "sentry_environment" ("organization_id", "name", "date_added") VALUES (%s, %s, %s) RETURNING "sentry_environment"."id"
)
>> Are you restoring from a backup of the same version of Sentry?
>> Are you restoring onto a clean database?
>> If so then this IntegrityError might be our fault, you can open an issue here:
>> https://github.com/getsentry/sentry/issues/new/choose
RpcImportError(
kind: IntegrityError,
on: InstanceID(model: 'sentry.environment'),
reason: UniqueViolation('duplicate key value violates unique constraint "sentry_environment_organization_id_name_95a37dc7_uniq"\nDETAIL: Key (organization_id, name)=(4, Production) already exists.\n')
SQL: INSERT INTO "sentry_environment" ("organization_id", "name", "date_added") VALUES (%s, %s, %s) RETURNING "sentry_environment"."id"
)
Traceback (most recent call last):
File "/.venv/bin/sentry", line 4, in <module>
raise SystemExit(main())
^^^^^^
File "/usr/src/sentry/src/sentry/runner/main.py", line 148, in main
func(**kwargs)
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/runner/decorators.py", line 35, in inner
return ctx.invoke(f, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/runner/commands/backup.py", line 764, in import_organizations
import_in_organization_scope(
File "/usr/src/sentry/src/sentry/backup/imports.py", line 541, in import_in_organization_scope
return _import(
^^^^^^^^
File "/usr/src/sentry/src/sentry/backup/imports.py", line 474, in _import
do_writes(pk_map)
File "/usr/src/sentry/src/sentry/backup/imports.py", line 403, in do_writes
do_write(import_write_context, pk_map, model_name, json_data, offset)
File "/usr/src/sentry/src/sentry/backup/imports.py", line 334, in do_write
raise ImportingError(result)
sentry.backup.imports.ImportingError: is_err=True kind='IntegrityError' on=InstanceID(model='sentry.environment', ordinal=None) left_pk=None right_pk=None reason='UniqueViolation(\'duplicate key value violates unique constraint "sentry_environment_organization_id_name_95a37dc7_uniq"\\nDETAIL: Key (organization_id, name)=(4, Production) already exists.\\n\')\nSQL: INSERT INTO "sentry_environment" ("organization_id", "name", "date_added") VALUES (%s, %s, %s) RETURNING "sentry_environment"."id"'
I was able to get sudo ./sentry-admin.sh import config backup.json and sudo ./sentry-admin.sh import users backup.json to work
but sudo ./sentry-admin.sh import organizations backup.json fails with the same UniqueViolation error even if I remove all the sentry.environment models from the backup.json file
More findings. after removing literally every model in the backup.json file... i kept getting the same error.
It's as if it was reading the backup.json file from somewhere else, so to test this theory I deleted the backup.json file and re-ran the import command.
I expected for it to fail since the backup.json file is missing but it still gave the same UniqueViolation error.
I've made sure there's no backup.json file in the root of the sentry directory or in the scripts folder.
I'm confused.
This appears to be pretty strange behavior. If this is a new install on a fresh postgres db these integrity errors should not be occurring. @azaslavsky would you be able to take a look at this when you have a chance?
Hi, also got this issue at version 23.5.1 but with different constraint that appears to this thread. My work around is to delete the constraint and add it again after the restore process done. Tried to reset the sequences of the migration didn't works for me
dug some links and might be give us some clue https://forum.djangoproject.com/t/django-db-utils-integrityerror-duplicate-key-value-violates-unique-key-constraint/15625 https://stackoverflow.com/questions/21458387/transactionmanagementerror-you-cant-execute-queries-until-the-end-of-the-atom
tl;dr
For some reason people start guessing the issue might be happen because how the transaction handled in django ORM and it's sequences so yeah it might be happen again to other people
loop in @hubertdeng123 and @azaslavsky maybe wanna review it again
i think i found the root cause,
somehow 1 email registered 3 times with different id, different hash, and different date_hash added. Is there any logic behind this?
This appears to be pretty strange behavior. If this is a new install on a fresh postgres db these integrity errors should not be occurring. @azaslavsky would you be able to take a look at this when you have a chance?
I'm getting the same behavior while using ./scripts/backup.sh global and ./scripts/restore.sh global
Migrating from Sentry 25.1.0 on CentOS 7 to Sentry 25.1.0 on Ubuntu 24.04.
On the Ubuntu server, it's a fresh install, nothing in the database. I had to restore only the config and the users like @edgariscoding mentionned. When trying to restore the organization I get:
RpcImportError(
kind: IntegrityError,
on: InstanceID(model: 'sentry.environmentproject'),
reason: UniqueViolation('duplicate key value violates unique constraint "sentry_environmentprojec_project_id_environment_i_91da82f2_uniq"\nDETAIL: Key (project_id, environment_id)=(24, 195) already exists.\n')
SQL: INSERT INTO "sentry_environmentproject" ("project_id", "environment_id", "is_hidden") VALUES (%s, %s, %s) RETURNING "sentry_environmentproject"."id"
)
>> Are you restoring from a backup of the same version of Sentry?
>> Are you restoring onto a clean database?
>> If so then this IntegrityError might be our fault, you can open an issue here:
>> https://github.com/getsentry/sentry/issues/new/choose
RpcImportError(
kind: IntegrityError,
on: InstanceID(model: 'sentry.environmentproject'),
reason: UniqueViolation('duplicate key value violates unique constraint "sentry_environmentprojec_project_id_environment_i_91da82f2_uniq"\nDETAIL: Key (project_id, environment_id)=(24, 195) already exists.\n')
SQL: INSERT INTO "sentry_environmentproject" ("project_id", "environment_id", "is_hidden") VALUES (%s, %s, %s) RETURNING "sentry_environmentproject"."id"
)
Traceback (most recent call last):
File "/.venv/bin/sentry", line 4, in <module>
raise SystemExit(main())
~~~~^^
File "/usr/src/sentry/src/sentry/runner/main.py", line 148, in main
func(**kwargs)
~~~~^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/.venv/lib/python3.13/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/src/sentry/src/sentry/runner/decorators.py", line 35, in inner
return ctx.invoke(f, *args, **kwargs)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.13/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/src/sentry/src/sentry/runner/commands/backup.py", line 766, in import_organizations
import_in_organization_scope(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
src,
^^^^
...<3 lines>...
printer=printer,
^^^^^^^^^^^^^^^^
)
^
File "/usr/src/sentry/src/sentry/backup/imports.py", line 542, in import_in_organization_scope
return _import(
src,
...<4 lines>...
printer=printer,
)
File "/usr/src/sentry/src/sentry/backup/imports.py", line 475, in _import
do_writes(pk_map)
~~~~~~~~~^^^^^^^^
File "/usr/src/sentry/src/sentry/backup/imports.py", line 404, in do_writes
do_write(import_write_context, pk_map, model_name, json_data, offset)
~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/backup/imports.py", line 335, in do_write
raise ImportingError(result)
sentry.backup.imports.ImportingError: is_err=True kind='IntegrityError' on=InstanceID(model='sentry.environmentproject', ordinal=None) left_pk=None right_pk=None reason='UniqueViolation(\'duplicate key value violates unique constraint "sentry_environmentprojec_project_id_environment_i_91da82f2_uniq"\\nDETAIL: Key (project_id, environment_id)=(24, 195) already exists.\\n\')\nSQL: INSERT INTO "sentry_environmentproject" ("project_id", "environment_id", "is_hidden") VALUES (%s, %s, %s) RETURNING "sentry_environmentproject"."id"'
I even ran reset.sh before trying to import.
Anyone has a workaround for this? Without the environments and projects, there is a lot of manual work to bring this back online.
For experimentation's sake, I delete all sentry.environmentproject and all sentry.environment from the backup and it continues to fail.
django.db.utils.IntegrityError: UniqueViolation('duplicate key value violates unique constraint "sentry_teamreplica_pkey"\nDETAIL: Key (id)=(1) already exists.\n')
SQL: INSERT INTO "sentry_teamreplica" ("team_id", "organization_id", "slug", "name", "status", "date_added", "org_role") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "sentry_teamreplica"."id"
Best I managed was to restore users, organizations and config.
Restoring global does not work and results in the Traceback about ImportingError.