New migration system in v3

[medihack]

Closes #1189

Introduce a new migration process: pre-migrations and post-migrations.

• They will be tied to the corresponding release
• Reworked v3 migrations with this new template in mind
• They allow us to provide a much saner path for blue-green deployment
• The process is currently not tested (but nor was the previous system)
• We will need to be careful when we do a release to rename the migrations in ongoing PRs (as today)
• We (hopefully) won't have to do multiple releases just for the sake of blue-green compatibility.
• I'm adding versioning in the name of all internal objects (functions, triggers, etc) (as mentionned here)

So the idea is that:

• Procrastinate v2 with the pre-migrations & without the post-migrations should work
• Procrastinate v3 with the pre-migrations & without the post-migrations should work
• Procrastinate v3 with all migrations should be what we currently have in v3

(I'm 80% sure that I have missed a few corner cases)

Successful PR Checklist:

• [ ] Tests
  • [ ] (not applicable?)
• [x] Documentation
  • [ ] (not applicable?)

PR label(s):

• [x] https://github.com/procrastinate-org/procrastinate/labels/PR%20type%3A%20breaking%20%F0%9F%92%A5
• [ ] https://github.com/procrastinate-org/procrastinate/labels/PR%20type%3A%20feature%20%E2%AD%90%EF%B8%8F
• [ ] https://github.com/procrastinate-org/procrastinate/labels/PR%20type%3A%20bugfix%20%F0%9F%95%B5%EF%B8%8F
• [ ] https://github.com/procrastinate-org/procrastinate/labels/PR%20type%3A%20miscellaneous%20%F0%9F%91%BE
• [ ] https://github.com/procrastinate-org/procrastinate/labels/PR%20type%3A%20dependencies%20%F0%9F%A4%96
• [ ] https://github.com/procrastinate-org/procrastinate/labels/PR%20type%3A%20documentation%20%F0%9F%93%9A

[github-actions[bot]]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
procrastinate
manager.py
Project Total

This report was generated by python-coverage-comment-action

[medihack]

@ewjoachim, in our session, I think we made a wrong assumption regarding function renaming. Function bodies are stored as strings, so when a function is renamed and used by another function, the function using the renamed function must be recreated. This is not true for triggers (or tables in case of used types) as those store the references and don't have to be recreated. So I did it in very small steps to make it comprehensible, and everything is renamed now.

[medihack]

Awesome work :) I'm still trying to find some time to test this with v2 & v3 with only pre. Life is a bit hectic at the moment so I can't commit on a date. Still, I'm very appreciative of your work :)

Just take your time. Let me know if there is anything specific I can help with. I will also look into how to test the pre schema with v2 and v3.

[medihack]

@ewjoachim Unfortunately, I have difficulty checking if the pre-migration runs with even v3. For testing purposes, I deleted the post-migration, ran all migrations on an empty database, and dumped the resulting schema with pg_dump. Then I replaced our schema.sql with that dump and ran the integration and acceptance tests, but I ended up with many (for me cryptic) errors. Was my approach correct? Have you had something different in mind? Do you perhaps have time for a video conference next week or so to solve this together?

[ewjoachim]

I'd love that :) let's schedule this together !