django-mysql
django-mysql copied to clipboard
adamchainz
Migrations via pt-online-schema-change
It may be possible to easily/for some cases convert the SQL output of django migrations into a series of pt-online-schema-change commands + mysql command to INSERT into migration history in a script.
Given my recent exploration into this locally sheds light on one particular issue atm.
Specifically with temp table constraint names surpassing the max length of 64 characters. This is not an issue for Djangos migrations framework but doesn't seem to play well with pt-osc. The best solution to this was installing the latest from Github which has PT-144 fixed but has not been released yet.
I think it would be a great feature to add. However, pt-osc is not beholden to Django's approach to schema changes. So, there could always be issues to address later on. Just some food for thought.
Thanks again for having this library.
Thanks for the info! I've used pt-osc to run migrations for some time on a Django app, but it's not using Django migrations :) Good to know what some of the issues will be. But when are you seeing 64 character constraint names? Do you have particularly long model names?
@adamchainz I don't have long model names. From what I saw. Django has it's own way of generating FK constraint names and when pt-osc decides to re-attach those constraints I see that it prepends it's own constraint name scheme to Django's name scheme. I don't have an example to put here but it's the combo of those two together that caused pt-osc's 64 character limit check to fail. I saw this mostly with FK constraints for a given table.
I am by no means an expert with pt-osc. I could be using it incorrectly and misinterpreting the error/warning output from the tool. Overall I still want it to work but haven't quite cracked that nut.
Ok well thanks for the info. Sounds like they'll have a fix though which is good.
@adamchainz Would there be any chance of pt-online-schema-change being to alter tables? Maybe even only on tables over a certain number of rows? Obviously this would need to be enabled in some way as we can't assume everyone has this tool installed. Thanks for the excellent package!
I don't use mysql atm but if you have a PR I will look at it. I think you'd want to override the database backend and its schema editor.
Ok, great, I'll take a look and see where I get too. Appreciate the quick response and you looking at a PR if I can get to the bottom of it 👍
Hi @violuke! we're also having great success with pt-online-schema-change and we're interested in using it with our migrations, as well. Shall we streamline our efforts?
Since I don't use MySQL currently, I don't think I'll be adding any automatic online migration support in the near future. If I return to it, I'll certainly investigate all the tools out there.
Not exactly using pt-online-schema-change, but solving many issues with adding columns to large tables via this PR: https://github.com/django/django/pull/17539. Hoping for your assistance @adamchainz :heart: