Issue importing sentry exported data

[MichelBitar99]

Self-Hosted Version

21.8.0

CPU Architecture

x86_64

Docker Version

20.10.18

Docker Compose Version

1.29.2

Steps to Reproduce

I am trying to migrate data (without event data) from sentry k8s to sentry self-hosted.

The export.json contains records similar to the following for all users:

[
{
    "model": "sentry.email",
    "pk": 3,
    "fields": {
      "email": "<user_example>",
      "date_added": "2021-02-25T13:30:14.337Z"
    }
  },
 {
    "model": "sentry.user",
    "pk": 3,
    "fields": {
      "password": "pbkdf2_sha256$120000$YUak................",
      "last_login": "2024-02-19T11:47:41.798Z",
      "username": "<user_example>",
      "name": "<user_name>",
      "email": "<user_example>",
      "is_staff": false,
      "is_active": true,
      "is_superuser": false,
      "is_managed": false,
      "is_sentry_app": null,
      "is_password_expired": false,
      "last_password_change": "2021-02-25T13:30:14.328Z",
      "flags": "0",
      "session_nonce": "Uxx....",
      "actor": 1,
      "date_joined": "2021-02-25T13:30:14.304Z",
      "last_active": "2024-03-04T08:26:57.695Z"
    }
  },
{
    "model": "sentry.useremail",
    "pk": 3,
    "fields": {
      "user": [
        "<user_example>"
      ],
      "email": "<user_example>",
      "validation_hash": "MXw1...............",
      "date_hash_added": "2021-02-25T13:30:14.334Z",
      "is_verified": false
    }
  }
]

I am running inside the web container the following command: sentry import export.json

Expected Result

I am expecting the data to be migrated successfully

Actual Result

The import is failing with the following error:

django.db.utils.IntegrityError: IntegrityError('duplicate key value violates unique constraint "sentry_useremail_user_id_email_ade975f1_uniq"\nDETAIL:  Key (user_id, email)=(3, <user_example>) already exists.\n',)
SQL: UPDATE "sentry_useremail" SET "user_id" = %s, "email" = %s, "validation_hash" = %s, "date_hash_added" = %s, "is_verified" = %s WHERE "sentry_useremail"."id" = %s

The only way I can fix this is by deleting all the useremail models as such: jq 'del(.[] | select(.model == "sentry.useremail"))' And then importing the data.

However, if I do that, I will have the following error under the Members tab:

Going a step further, even if we decide to lose all users and create them again, we would face issues restoring the data because of "sentry.userip", and then dashboards' created_by field etc...

[azaslavsky]

This script was basically nonfunctional on old versions, prior to 23.11.1. I'd recommend using the approach specified here for your version instead. Conversely, you can upgrade to >=23.11.1 (don't forget the hard stop at 23.6.2 though!), at which point new backups generated post-upgrade will be able to be restored properly (however, this backup, which was made using an old version, will still fail). Sorry, this wasn't a great experience before we revamped it in 23.11.1, so the options for old versions aren't great!

[MichelBitar99]

This script was basically nonfunctional on old versions, prior to 23.11.1. I'd recommend using the approach specified here for your version instead. Conversely, you can upgrade to >=23.11.1 (don't forget the hard stop at 23.6.2 though!), at which point new backups generated post-upgrade will be able to be restored properly (however, this backup, which was made using an old version, will still fail). Sorry, this wasn't a great experience before we revamped it in 23.11.1, so the options for old versions aren't great!

Thank you for your reply. Unfortunately updating to 23.6.2 then to 23.11.1 involves a lot of overhead. Would there be a way to "hack" our way into it?

After removing from the exported json file the sentry.useremail data, I was able to import it. However, visiting the members page is failing, and I can see the following error on the web container logs:

| Traceback (most recent call last): web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/base.py", line 125, in handle_exception web_1 | response = super().handle_exception(exc) web_1 | File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 465, in handle_exception web_1 | self.raise_uncaught_exception(exc) web_1 | File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception web_1 | raise exc web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/base.py", line 241, in dispatch web_1 | response = handler(request, *args, **kwargs) web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/endpoints/organization_member_index.py", line 193, in get web_1 | paginator_cls=OffsetPaginator, web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/base.py", line 340, in paginate web_1 | results = on_results(cursor_result.results) web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/endpoints/organization_member_index.py", line 190, in <lambda> web_1 | expand=expand web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/serializers/base.py", line 85, in serialize web_1 | return [serializer(o, attrs=attrs.get(o, {}), user=user, **kwargs) for o in objects] web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/serializers/base.py", line 85, in <listcomp> web_1 | return [serializer(o, attrs=attrs.get(o, {}), user=user, **kwargs) for o in objects] web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/serializers/base.py", line 97, in __call__ web_1 | return self.serialize(obj, attrs, user, **kwargs) web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/api/serializers/models/organization_member.py", line 107, in serialize web_1 | "roleName": roles.get(obj.role).name, web_1 | File "/usr/local/lib/python3.6/site-packages/sentry/roles/manager.py", line 50, in get web_1 | return self._roles[id] web_1 | KeyError: 'read_only_member' web_1 | 13:41:23 [ERROR] django.request: Internal Server Error: /api/0/organizations/***/members/ (status_code=500 request=<WSGIRequest: GET '/api/0/organizations/****/members/'>)

Would there be some manual operations on the DB for example that would solve that issue?

[azaslavsky]

It's very hard to say. The reason this operation was so buggy was because it just exported/imported raw tables. But a new instance has entries of its own - these often collided, causing pks that point to nowhere (since the auto_increment is off), which produces all sorts of state dependent bugs. It's very hard to remotely debug this issue, especially on an old version like this.

Have you considered doing a full volume backup instead?

[MichelBitar99]

I see, I tried fixing the issue manually but got nowhere.

I believe that backing up the volumes is going to be much more complicated as there are a lot of discrepancies between the k8s and self-hosted version. I would bump into much more issues. One simple example is that k8s uses bitnami/postgresql , and is on a whole different version of postgres.

Will try upgrading to 23.11.1 then performing a backup/restore.

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀