django-mysql
django-mysql copied to clipboard
adamchainz
Test MariaDB 10.2, activate JSONField for it
Summary: It looks like MariaDB 10.2 supports the JSON type ( https://mariadb.com/kb/en/mariadb/json-functions/ ) and also more goodness ( https://mariadb.com/kb/en/mariadb/what-is-mariadb-102/ ). We should start testing on it, and check if JSONField can work on it (its system check + docs will then need updating).
I've had a brief look. To be truly compatible we'd need to also add a check to the column, like CHECK JSON_VALID(fieldname), so that inserting invalid JSON isn't possible - this is what the MariaDB docs advise. Django doesn't support constraints like this at the moment, let alone them coming from fields, but it's worth looking at before doing this.
If you want to add it TODAY and don't mind adding the CHECK yourself, you could subclass the built-in JSONField and disable the check that we're on Oracle MySQL 5.7, that should be enough to let you use it.
Triggers aren't quite the same as CHECK constraints, see https://mariadb.com/kb/en/mariadb/constraint/#check-constraint-expressions . Neither triggers nor CHECK constraints have any code representation in Django which is what makes them challenging to add.
@adamchainz I would like to use django-mysql with MariaDB 10.2+ and query JSON fields, is it currently possible? thank you.
@afausti I'm afraid not, as otherwise this ticket would have been closed :)
So I've just realized I've confused column checks and table checks. Django already supports column checks - retrieved from https://github.com/django/django/blob/master/django/db/models/fields/init.py#L667 - and seems to apply them even on backends where supports_column_check_constraints is False (like the MySQL one, for now). So we can go ahead and use them :)
MariaDB 10.2 also supports json as a data type, but only as an alias for text. So we'll add the json_valid check constraint for both MySQL (where it will be parsed but not applied) and MariaDB.
@adamchainz I also would like to use the JSON fields in our application (we are running Django 1.9.2 with MariaDB 10.2.10). Whenever I try to use it, I get the following error during migrations:
SystemCheckError:
System check identified some issues:
ERRORS: controller.Device.properties: (django_mysql.E016) MySQL 5.7+ is required to use JSONField
Is this because the django-mysql package only checks MySQL version > 5.7?
Kind regards,
Diederik
@diedoman until this issue is closed, JSONField is only supported on MySQL 5.7. You can always use SILENCED_SYSTEM_CHECKS and use it in 'unsupported' mode - it won't be enforcing JSON is valid at the DB level. Or you can use DynamicField.
So I've tried the 'unsupported' mode with MariaDB 10.2.15. I've also tested adding the json_valid check constraint (using db_check method) and it mostly worked. There is a problem with path lookups though. The SQL produced contains CAST(%s AS JSON) and MariaDB doesn't like it (anymore? https://jira.mariadb.org/browse/MDEV-11439).
Apart from sharing my experience, I think I'd like to ask if there are any plans to enable JSONField for MariaDB in a 'supported' mode? ;)
I started a branch to work on this back in November-ish, I found the same problem you're reporting. I will have another look at Djangocon EU this week.
Is there any update? I need to use JsonField() with MariaDB 10.4.0 and Django 2.1.7.
There's no update, again if there was it would be here. I have limited time to work on open source. From my initial experimetns, I recall this could require a good few hours to tackle. If you want to sponsor development of this feature I'm open to that - email [email protected] for more details.
In other news there's a Google Summer of Code student working on Django to bring a JSONField to core: https://summerofcode.withgoogle.com/organizations/5465143218012160/#projects / https://code.djangoproject.com/wiki/SummerOfCode2019 . This will bring in some code from this library and if all goes well it will be available in Django 3.0.
Unfortunately, Django 3.0 has no support for MySQL/MariaDB specific fields.
What's your vision about this now? :)
I don't have time to work on this right now (at least unpaid).
My main vision would be to spend time getting the universal JSONField merged to Django core. After that's done, I'd be open to making django-jsonfield, or perhaps a new package with a new name, a copy of that universal field backported to older Django versions.
So I've tried the 'unsupported' mode with MariaDB 10.2.15. I've also tested adding the
json_validcheck constraint (usingdb_checkmethod) and it mostly worked. There is a problem with path lookups though. The SQL produced containsCAST(%s AS JSON)and MariaDB doesn't like it (anymore? https://jira.mariadb.org/browse/MDEV-11439).Apart from sharing my experience, I think I'd like to ask if there are any plans to enable JSONField for MariaDB in a 'supported' mode? ;)
I just hit this as well when working on a new app after choosing Django and MariaDB. IMHO the installation section (https://django-mysql.readthedocs.io/en/latest/installation.html) listing MariaDB as tested and supported is a bit misleading as one would expect it to works the same as MySQL, would be nice to at least add some precaution on using JSON column.
@ronaldchoi The JSONField docs state:
This field requires Django 1.8+ and MySQL 5.7+.
I'd accept a PR adding a sentence to the installation instructions that not all features are supported/tested on all database versions.
I also get this message: "MySQL 5.7+ is required to use JSONField".
My env: Python 3.7.7, Django 3.0.7, MariaDB 10.5.3, mysqlclient 2.0.0, django-mysql 3.7.1.
Any solution to create a json field in MariaDB?
Django 3.1 in beta has a univresal JSONField, the dev who made it is working on a backport package: https://twitter.com/laymonage/status/1278485132765085696