sqlacodegen icon indicating copy to clipboard operation
sqlacodegen copied to clipboard

Fixed omission of 'collation' keyword in existing CHAR type

Open hyoj0942 opened this issue 1 year ago • 8 comments

Fixes #285 Error in Generated SQLAlchemy Model for CHAR Column

hyoj0942 avatar Sep 25 '23 04:09 hyoj0942

Coverage Status

coverage: 97.644% (+0.005%) from 97.639% when pulling ac1bdb048304d14c33a105ebbadcb73fba7fd437 on hyoj0942:render_column_type into c68d2a8a0ec3511afb96983bdce6e2498aaa72b3 on agronholm:master.

coveralls avatar Sep 25 '23 04:09 coveralls

I do wonder if this problem is specific to only a single column type? I'm not very fond of special casing CHAR.

agronholm avatar Sep 25 '23 08:09 agronholm

A better solution would be inspecting the types and checking which parameters need to be keyword arguments.

agronholm avatar Sep 25 '23 08:09 agronholm

in my DB setup, almost all kinds are there and when I tried it, all types other than CHAR were error-free so, I started by merely changing the CHAR method that was responsible for the problem I'll look at the types that need the keyword, make any necessary edits, and then commit

hyoj0942 avatar Sep 25 '23 08:09 hyoj0942

I'll look at the types that need the keyword, make any necessary edits, and then commit

Rather than special casing specific types, it would be best to detect this automatically so it's more future proof. Plus there could be obscure vendor specific types that also need this treatment.

agronholm avatar Sep 25 '23 09:09 agronholm

I'll look at the types that need the keyword, make any necessary edits, and then commit

Rather than special casing specific types, it would be best to detect this automatically so it's more future proof. Plus there could be obscure vendor specific types that also need this treatment.

maybe there was a problem with importing multiple times

among them, the CHAR class in sqlalchemy/dialects/mysql/types.py caused an error while receiving an instance variable as a keyword argument. It works normally because the argument is received as a CHAR class in sqlalchemy/sql/sqltypes.py.

In SQLAlchemy

# sqlalchemy/sql/sqltypes.py
...
class String(Concatenable, TypeEngine[str]):

    """The base for all string and character types.

    In SQL, corresponds to VARCHAR.

    The `length` field is usually required when the `String` type is
    used within a CREATE TABLE statement, as VARCHAR requires a length
    on most databases.

    """

    __visit_name__ = "string"

    def __init__(
        self,
        length: Optional[int] = None,
        collation: Optional[str] = None,
    ):
...
...
class CHAR(String):

    """The SQL CHAR type."""

    __visit_name__ = "CHAR"
# sqlalchemy/dialects/mysql/types.py
...
class CHAR(_StringType, sqltypes.CHAR):
    """MySQL CHAR type, for fixed-length character data."""

    __visit_name__ = "CHAR"

    def __init__(self, length=None, **kwargs):
...

Generated file

from sqlalchemy import CHAR ...
from sqlalchemy.dialects.mysql import CHAR ...
...

hyoj0942 avatar Sep 27 '23 08:09 hyoj0942

How is it importing two different CHARs? Are the column types on the DB side different somehow?

agronholm avatar Sep 27 '23 18:09 agronholm

Just dropping in, having tried 3.0.0rc5 against our MySQL schema.

  • I am seeing the double CHAR import as above. Most of the imports don't collide because of the mysql CAPS obsession.
  • I am also seeing CHAR(16, 'utf8_unicode_ci') on test run, which is invalid in sqlalchemy.dialects.mysql - it doesn't accept a collation.
  • Finally I'm seeing some exciting errors around DOUBLE column types: E sqlalchemy.exc.ArgumentError: You must specify both precision and scale or omit both altogether.

One of our students had a bash at fixing sqlacodegen v2 here: https://github.com/wtsi-npg/sqlacodegen/tree/additional-patches , but the big sqlalchemy v2 upheaval made that moot.

nerdstrike avatar Apr 17 '24 16:04 nerdstrike