directus-sync icon indicating copy to clipboard operation
directus-sync copied to clipboard

Invalid payload. Collection \"logs\" doesn't exist.

Open BorisKamp opened this issue 1 year ago • 12 comments

Hey!

I pulled my config locally, I then pushed to master, which builds my docker production image, I then run docker compose exec -u root directus npx directus-sync diff which works, this is the output:

[13:32:01.623] INFO (119): [config] No config file found. Tried path: 
- /directus/directus-sync.config.js
- /directus/directus-sync.config.cjs
- /directus/directus-sync.config.json
[13:32:02.355] INFO (119): [snapshot] Found 0 change in collections
[13:32:02.355] INFO (119): [snapshot] Found 11 changes in fields
[13:32:02.355] INFO (119): [snapshot] Found 4 changes in relations
[13:32:02.731] INFO (119): [settings] Dangling id maps: 0 item(s)
[13:32:02.731] INFO (119): [settings] To create: 0 item(s)
[13:32:02.731] INFO (119): [settings] To update: 0 item(s)
[13:32:02.731] INFO (119): [settings] To delete: 0 item(s)
[13:32:02.731] INFO (119): [settings] Unchanged: 1 item(s)
[13:32:02.828] INFO (119): [folders] Dangling id maps: 0 item(s)
[13:32:02.828] INFO (119): [folders] To create: 0 item(s)
[13:32:02.828] INFO (119): [folders] To update: 0 item(s)
[13:32:02.828] INFO (119): [folders] To delete: 0 item(s)
[13:32:02.828] INFO (119): [folders] Unchanged: 0 item(s)
[13:32:02.930] INFO (119): [translations] Dangling id maps: 0 item(s)
[13:32:02.930] INFO (119): [translations] To create: 0 item(s)
[13:32:02.930] INFO (119): [translations] To update: 0 item(s)
[13:32:02.931] INFO (119): [translations] To delete: 0 item(s)
[13:32:02.931] INFO (119): [translations] Unchanged: 0 item(s)
[13:32:03.031] INFO (119): [flows] Dangling id maps: 0 item(s)
[13:32:03.031] INFO (119): [flows] To create: 0 item(s)
[13:32:03.031] INFO (119): [flows] To update: 0 item(s)
[13:32:03.031] INFO (119): [flows] To delete: 0 item(s)
[13:32:03.031] INFO (119): [flows] Unchanged: 0 item(s)
[13:32:03.130] INFO (119): [operations] Dangling id maps: 0 item(s)
[13:32:03.130] INFO (119): [operations] To create: 0 item(s)
[13:32:03.130] INFO (119): [operations] To update: 0 item(s)
[13:32:03.130] INFO (119): [operations] To delete: 0 item(s)
[13:32:03.130] INFO (119): [operations] Unchanged: 0 item(s)
[13:32:03.647] INFO (119): [roles] Dangling id maps: 0 item(s)
[13:32:03.647] INFO (119): [roles] To create: 1 item(s)
[13:32:03.647] INFO (119): [roles] To update: 1 item(s)
[13:32:03.647] INFO (119): [roles] To delete: 0 item(s)
[13:32:03.647] INFO (119): [roles] Unchanged: 2 item(s)
[13:32:04.368] INFO (119): [policies] Dangling id maps: 0 item(s)
[13:32:04.368] INFO (119): [policies] To create: 1 item(s)
[13:32:04.368] INFO (119): [policies] To update: 2 item(s)
[13:32:04.368] INFO (119): [policies] To delete: 0 item(s)
[13:32:04.368] INFO (119): [policies] Unchanged: 2 item(s)
[13:32:13.033] INFO (119): [permissions] Dangling id maps: 1 item(s)
[13:32:13.033] INFO (119): [permissions] To create: 23 item(s)
[13:32:13.033] INFO (119): [permissions] To update: 2 item(s)
[13:32:13.033] INFO (119): [permissions] To delete: 0 item(s)
[13:32:13.033] INFO (119): [permissions] Unchanged: 64 item(s)
[13:32:13.129] INFO (119): [dashboards] Dangling id maps: 0 item(s)
[13:32:13.129] INFO (119): [dashboards] To create: 0 item(s)
[13:32:13.129] INFO (119): [dashboards] To update: 0 item(s)
[13:32:13.129] INFO (119): [dashboards] To delete: 0 item(s)
[13:32:13.129] INFO (119): [dashboards] Unchanged: 0 item(s)
[13:32:13.235] INFO (119): [panels] Dangling id maps: 0 item(s)
[13:32:13.235] INFO (119): [panels] To create: 0 item(s)
[13:32:13.235] INFO (119): [panels] To update: 0 item(s)
[13:32:13.235] INFO (119): [panels] To delete: 0 item(s)
[13:32:13.235] INFO (119): [panels] Unchanged: 0 item(s)
[13:32:13.326] INFO (119): [presets] Dangling id maps: 0 item(s)
[13:32:13.326] INFO (119): [presets] To create: 0 item(s)
[13:32:13.326] INFO (119): [presets] To update: 0 item(s)
[13:32:13.326] INFO (119): [presets] To delete: 0 item(s)
[13:32:13.326] INFO (119): [presets] Unchanged: 0 item(s)
[13:32:13.326] INFO (119): ✅  Done!

I then run docker compose exec -u root directus npx directus-sync push which errors out:

[13:32:55.671] INFO (153): [config] No config file found. Tried path: 
- /directus/directus-sync.config.js
- /directus/directus-sync.config.cjs
- /directus/directus-sync.config.json
[13:32:56.156] INFO (153): ---- Push schema ----
[13:32:56.823] ERROR (153):
    errors: [
      {
        "message": "Invalid payload. Collection \"logs\" doesn't exist.",
        "extensions": {
          "reason": "Collection \"logs\" doesn't exist",
          "code": "INVALID_PAYLOAD"
        }
      }
    ]
    response: {}

logs is definitely there, here is a screenshot of my production db table called logs: Screenshot 2025-01-10 at 14 33 58 In directus_collections you can also see it: Screenshot 2025-01-10 at 14 34 50

Why am I getting this error? Is there anything else I can do to debug further?

BorisKamp avatar Jan 10 '25 13:01 BorisKamp

Hi @BorisKamp.

  • Do you see the collection logs in the target Directus instance models ?
  • Did you try to use the --force flag (npx directus-sync push --force) ?

EdouardDem avatar Jan 10 '25 14:01 EdouardDem

Hi @BorisKamp.

  • Do you see the collection logs in the target Directus instance models ?
  • Did you try to use the --force flag (npx directus-sync push --force) ?

Hey! Thanks for the reply. Can you clarify this:

Do you see the collection logs in the target Directus instance models

please? Thanks

BorisKamp avatar Jan 10 '25 16:01 BorisKamp

Sorry, I didn't see the screenshot of directus_collections. Can you run the command with the debug flag : npx directus-sync push --debug and also the logs from Directus ?

EdouardDem avatar Jan 11 '25 02:01 EdouardDem

Do you use folders or sub-models to organize you models in Directus ? This may be an issue with the snapshot restore process of Directus. Sometimes it tries to create the sub-model before the parent and it can lead to this error.

EdouardDem avatar Jan 11 '25 02:01 EdouardDem

Sorry, I didn't see the screenshot of directus_collections. Can you run the command with the debug flag : npx directus-sync push --debug and also the logs from Directus ?

See here:

root@cebv:/home/cebv/docker/cebv-portal-directus# docker compose exec -u root directus npx directus-sync push --debug
[09:14:40.471] INFO (1244): [config] No config file found. Tried path: 
- /directus/directus-sync.config.js
- /directus/directus-sync.config.cjs
- /directus/directus-sync.config.json
[09:14:40.851] DEBUG (1244): [migration-client] Directus 11.3.5 is compatible
[09:14:40.900] DEBUG (1244): [migration-client] Cache cleared
[09:14:41.032] INFO (1244): ---- Push schema ----
[09:14:41.717] ERROR (1244):
    errors: [
      {
        "message": "Invalid payload. Collection \"logs\" doesn't exist.",
        "extensions": {
          "reason": "Collection \"logs\" doesn't exist",
          "code": "INVALID_PAYLOAD"
        }
      }
    ]
    response: {}

BorisKamp avatar Jan 13 '25 09:01 BorisKamp

Do you use folders or sub-models to organize you models in Directus ? This may be an issue with the snapshot restore process of Directus. Sometimes it tries to create the sub-model before the parent and it can lead to this error.

I don't think so, these are my models: Screenshot 2025-01-13 at 10 15 37

BorisKamp avatar Jan 13 '25 09:01 BorisKamp

@EdouardDem I do recall months ago we went from the model name log to logs. I think this initiated the problem. What I don't get, is that now everything is renamed, there's still an issue.....

I removed the logs model entirely from production and then released a new docker image with the latest config from local (which includes the logs model), I then tried the push without success (see the error). However, I also created a snapshot (directus native) locally and applied that on production, and that worked great....

Maybe that helps you track the cause?

BorisKamp avatar Jan 13 '25 09:01 BorisKamp

@EdouardDem any idea?

BorisKamp avatar Jan 15 '25 15:01 BorisKamp

Maybe you have a remaining log reference in the JSON files. Can you search for log(not logs) in the directus-config folder ?

EdouardDem avatar Jan 16 '25 18:01 EdouardDem

Hey there, I have the same issue. I had a collection named "adress," which I deleted due to a typo. After that, I tried to sync my staging and production environments, but I encountered the same issue as you.

I connected directly to my Directus instance's database and found that there were still references to the deleted "adress" table in two Directus tables: directus_fields and directus_relations. I deleted these references to the non-existent table, and after that, my migration worked perfectly.

Josiyahse avatar Feb 25 '25 16:02 Josiyahse

Hey there, I have the same issue. I had a collection named "adress," which I deleted due to a typo. After that, I tried to sync my staging and production environments, but I encountered the same issue as you.

I connected directly to my Directus instance's database and found that there were still references to the deleted "adress" table in two Directus tables: directus_fields and directus_relations. I deleted these references to the non-existent table, and after that, my migration worked perfectly.

I do not have entries to the old collection log.

BorisKamp avatar Mar 25 '25 12:03 BorisKamp

Maybe you have a remaining log reference in the JSON files. Can you search for log(not logs) in the directus-config folder ?

I do not find any results when doing this @EdouardDem Image

BorisKamp avatar Mar 25 '25 12:03 BorisKamp