warehouse icon indicating copy to clipboard operation
warehouse copied to clipboard

Migrate keywords data structure to Postgres ARRAY type

Open miketheman opened this issue 2 years ago • 2 comments

This will be taken apart and shipped in parts.

🚨 Note: Data migration contained within

Does not drop existing column, but removes references to it.

Once merged, no new Releases will populate their keywords column, only the keywords_array column will be filled.

miketheman avatar Feb 15 '23 02:02 miketheman

I haven't read the entire PR yet, but you probably want this split over multiple deploys, currently what's going to happen is:

  1. Column will be added, data migration will be run.
  2. New deployment will occur with the new code.
  3. Old deployment will be spun down, shutting down old code.

The important bit of that is that the old code will still be running after the data migration has completed, so anything uploaded in that window of time via the old code will have keywords populated, but not keywords_array.

So I would refactor this into multiple PRs, that we can deploy in separate deployments:

  1. Adds the keywords_array column, sets up the upload code to write both columns.
  2. Does the data migration, guarding not to run against rows that (1) would have already written, sets up the code to only use the keywords_array column.
    • Optional: This could also drop the keywords column from the code, but not from the database, making SQLAlchemy completely ignore that column.
  3. If the keywords column has been dropped from the code, drop it from the database as well.

dstufft avatar Feb 15 '23 13:02 dstufft

probably want this split over multiple deploys

Yep! Going to dismantle and rebuild, now that all the pieces are known.

miketheman avatar Feb 15 '23 15:02 miketheman