migration-tool icon indicating copy to clipboard operation
migration-tool copied to clipboard

fix: relationship payload data is mapped to V9 endpoint

Open hakan-erdem-temiz opened this issue 3 years ago • 5 comments

V9 relation endpoint only accepts the object, therefore relation data mapped according to V9 endpoint.

hakan-erdem-temiz avatar Jun 11 '21 08:06 hakan-erdem-temiz

@hakan-erdem-temiz would you be willing to pull my commit into this PR also? https://github.com/directus-community/migration-tool/commit/14758dbaff140514db82366059037dc1d55b98a1

It builds off your work. Any collections that linked to files and users were not uploading correctly because of the missing meta prop.

dwene avatar Jul 08 '21 23:07 dwene

@dwene Did you get it work successfully? This PR doesn't seem to fix the migration of relations properly. I'm still encountering errors with it :( Trying to debug is quite tedious as the v8 instance I want to migrate has like 800 files attached to it (so it's not that simple to always rerun (or just takes immense time...)

moekify avatar Jul 10 '21 11:07 moekify

@moekify I also had about 800 files to migrate. Eventually I did get it to work but I had to debug and modify this script a LOT and run the migration in pieces.

  1. I had to go into my v8 instance and change all the join columns that linked to the directus_files id to be char(36) columns. Directus v9 uses UUID for all the files, and it will fail to setup relations if you don't have those as char(36).
  2. I also had to ensure that all the join columns in my v8 instance between collections were the same type. I had a few columns like team_member_id that were varchar(256) that pointed to another collection.id (int primary key). The new directus adds more database indices so those types need to be identical.
  3. A huge help was turning up the logging level on my directus 9 version. Then directus 9 would spit out the bad SQL commands in it's logs and I could determine what was wrong. Also increasing the MAX_PAYLOAD_SIZE size past 100kb (the default).
  4. Another helpful thing was writing the context to a file so I could see what all it was trying to do, and have it pickup where it left off (without deleting my database and starting over).
  5. In the end there were 2 bad relations coming from directus 8 that I just had to ignore. One was trying to join 2 collections together using a status column, and I couldn't figure out where it was coming from. So I tweaked the script to ignore those 2 relations once I fixed all the other issues.

Hopefully that helps some! Here's my sandbox code, it definitely will not work out of the box for you, but it shows where I write the context to a JSON file and read it back, so I could migrate everything in pieces.

https://github.com/dwene/migration-tool/pull/1

dwene avatar Jul 10 '21 13:07 dwene

@moekify we should update the script to save the state to a file every step of the way, so it can restart from a point halfway instead of having to start from the beginning each time

edit: whoops, just saw that's exactly what @dwene is talking about 👍🏻

rijkvanzanten avatar Jul 11 '21 06:07 rijkvanzanten

@moekify we should update the script to save the state to a file every step of the way, so it can restart from a point halfway instead of having to start from the beginning each time

edit: whoops, just saw that's exactly what @dwene is talking about 👍🏻

I tried to write this up. #37 Also @dwene ty for your fix of the meta checks for data migrations :)

moekify avatar Jul 13 '21 16:07 moekify