django-cassandra-engine icon indicating copy to clipboard operation
django-cassandra-engine copied to clipboard

unable to create or update object with user defined type

Open andy-io opened this issue 8 years ago • 5 comments

I am unable to perform an update on a user defined type after an object is created.

class address(UserType):
    street = columns.Text()
    zipcode = columns.Integer()


class users(DjangoCassandraModel):
    name = columns.Text(primary_key=True)
    addr = columns.UserDefinedType(address)

users.objects.create(name="Andy")
u = users.objects.get(name="Andy")
u.addr = address(street="Easy St.", zipcode=99999)
u.save()

cassandra.protocol.SyntaxException: <Error from server: code=2000 [Syntax error in CQL query] message="line 1:49 no viable alternative at input ',' (...SET "addr" = {'zipcode': [None],...)">

u = users.objects.create(name='Andy', addr=address(street='my street', zipcode=5678))

I get the same error for a direct creation as well.

andy-io avatar Dec 24 '16 11:12 andy-io

Have you fixed it yet?

Rot37 avatar Jun 01 '17 10:06 Rot37

Got the same issue.

ayasynetskyi avatar Apr 21 '18 08:04 ayasynetskyi

no fix yet, i discontinued using it

andy-io avatar Apr 29 '18 16:04 andy-io

Switched to Spring. I always regret trying to use python for anything.

simeon4110 avatar Apr 30 '18 15:04 simeon4110

Highlighting @r4fek

Found a fix for this after some digging, the issue here appears to be mainly a syntactical one. See: https://docs.datastax.com/en/developer/python-driver/3.19/user_defined_types/

As shown in the code example, inserting data for UDT columns without registering a class works fine for prepared statements. However, you must register a class to insert UDT columns with unprepared statements.* You can still query UDT columns without registered classes using unprepared statements, they will simply return namedtuple instances (just like prepared statements do).

Without some assistance, the underlying python-driver interprets your unregistered UserType models as plain dictionaries and throws a fit. The fix is to register those models so that:

A) They're serialized properly into the literal: 'value' format when inserted B) They're deserialized into a proper dict, rather than a series of tuples when selected

To test the first part there, you can do the following and add it to your UserType models to verify that it inserts properly where it wouldn't before:

class MonkeyPatchedUDT():
  def __str__(self):
    """Overrides UDT string representations to fix a query formatting issue.
    There are three (known) issues with the way that UserTypes are translated into CQL syntax:

    1) None values are not converted to `null`
    2) Keys are surrounded by 'quotes', which CQL does not like
    3) String values are printed as literals rather than 'quoted'
    """

    # Grab the original string representation.
    string_repr = super().__str__()

    # Replace all None values with 'null' to satisfy problem #1
    string_repr = string_repr.replace('None', 'null')

    for key in self.keys():
      # Remove the quotes around all stringified keys to satisfy problem #2
      string_repr = string_repr.replace('\'{}\''.format(key), key)
      value = getattr(self, key)

      # Is this a string field?
      field_is_string = isinstance(value, str)

      # Place quotes around string values to satisfy problem #3
      if field_is_string:
        string_repr = string_repr.replace(value, '\'{}\''.format(value))

    return string_repr

That is NOT the fix, however. The real fix is to call cluster.register_user_type('somekeyspace', 'some_user_defined_type', MyUserDefinedType) for every UserType whenever and wherever a connection is initiated. Has to be done every time for every UserType, that setting does not persist.

In theory this library would have to loop over every known UserType and register it before returning a cluster here.

The band-aid for that is to add the following where you define your models:

from django.db import connections
from cassandra.cluster import UserTypeDoesNotExist

user_types = [ SomeType, OtherType, SuperType ]
database_name = 'cassandra'
connection = connections[database_name]

for udt in user_types:
  try:
    connection.connection.cluster.register_user_type(database_name, udt.type_name(), udt)
  except UserTypeDoesNotExist:
    pass

NoiSek avatar Mar 06 '20 04:03 NoiSek