django-cockroachdb
django-cockroachdb copied to clipboard
add SmallAutoField support
Django 3.0 adds support for SmallAutoField, however, cockroachdb's unique_rowid()
generates 8-byte values that are too large for int2
(2-byte). I'm not sure how to proceed.
Test failure:
======================================================================
ERROR: test_fk_to_smallautofield (many_to_one.tests.ManyToOneTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.NumericValueOutOfRange: integer out of range for type int2 (column "country_id")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/tim/code/django/tests/many_to_one/tests.py", line 583, in test_fk_to_smallautofield
City.objects.create(country=us, name='Chicago')
File "/home/tim/code/django/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/tim/code/django/django/db/models/query.py", line 433, in create
obj.save(force_insert=True, using=self.db)
File "/home/tim/code/django/django/db/models/base.py", line 746, in save
force_update=force_update, update_fields=update_fields)
File "/home/tim/code/django/django/db/models/base.py", line 784, in save_base
force_update, using, update_fields,
File "/home/tim/code/django/django/db/models/base.py", line 886, in _save_table
results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
File "/home/tim/code/django/django/db/models/base.py", line 925, in _do_insert
using=using, raw=raw,
File "/home/tim/code/django/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/tim/code/django/django/db/models/query.py", line 1204, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
File "/home/tim/code/django/django/db/models/sql/compiler.py", line 1344, in execute_sql
cursor.execute(sql, params)
File "/home/tim/code/django/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/home/tim/code/django/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
django.db.utils.DataError: integer out of range for type int2 (column "country_id")
smallserial
is used for SmallAutoField
:
CREATE TABLE "smallauto_country" ("id" smallserial NOT NULL PRIMARY KEY, "name" varchar(50) NOT NULL);
However, it looks like cockroachdb uses unique_rowid()
for that column. The 8-byte id is stored without error despite the fact that smallserial is 2-byte on PostgreSQL . The error comes when using the large id in a foreign key (which uses smallint
):
CREATE TABLE "smallauto_city" ("id" bigserial NOT NULL PRIMARY KEY, "name" varchar(50) NOT NULL, "country_id" smallint NULL);
I wonder if we should change the smallserial
alias in CockroachDB so it uses a different function that generates a smaller int? Like implement a unique_smallrowid()
function, perhaps. I don't know if there are other implications with doing that.
FWIW, there also is a cluster setting called serial_normalization
that allows the SERIAL type to use a sequence instead of unique_rowid()
. I don't think we want to use it, but just listing it as an option. Details on the setting:
https://www.cockroachlabs.com/docs/releases/v2.1.0-beta.20180827.html
https://www.cockroachlabs.com/docs/stable/cluster-settings.html
cc @jordanlewis @apantel
I'm guessing that unique_smallrowid()
might be the way to go since CockroachDB's documentation says, "SERIAL is provided only for compatibility with PostgreSQL. New applications should use real data types and a suitable DEFAULT expression."
Django's AutoField
and BigAutoField
are both using integer
data type with DEFAULT unique_rowid()
rather than serial
and bigserial
.