sqlalchemy icon indicating copy to clipboard operation
sqlalchemy copied to clipboard

MySQL DOUBLE accepts deprecated params that dont propagate from the base DOUBLE type, document this caveat

Open MacBath opened this issue 11 months ago • 3 comments

Describe the bug

Hi im trying to get startet with sql alchemy and alembic and just found a bug.

during the translation of a double precision=17, and decimal_return_scale=7 into mysql i always get an exception. sqlalchemy.exc.ArgumentError: You must specify both precision and scale or omit both altogether.

When i check the kw at dialects/mysql/types.py i see that the parameter there is still the parameter decimal_return_scale=7, but named argument scale is None.

Optional link from https://docs.sqlalchemy.org which documents the behavior that is expected

No response

SQLAlchemy Version in Use

2.0.28

DBAPI (i.e. the database driver)

mysqlclient

Database Vendor and Major Version

MariaDB 10.*

Python Version

3.11

Operating system

Windows

To Reproduce

from sqlalchemy.orm import DeclarativeBase, MappedAsDataclass, Mapped, mapped_column
from sqlalchemy.types import DOUBLE
from datetime import datetime
from sqlalchemy import create_engine


class Base(MappedAsDataclass, DeclarativeBase):
    """Base Class for ORM Table Modell"""

    pass


metadata = Base.metadata


class mytable(Base):
    __tablename__ = "tablename"

    _unix_dt: Mapped[float] = mapped_column(
        type_=DOUBLE(precision=17, decimal_return_scale=7),
        primary_key=True,
    )
    _dt: Mapped[datetime] = mapped_column(nullable=True, server_default=None)


def main():
    engine = create_engine("mysql+mysqldb://root:secret@localhost/dummdb")
    Base.metadata.create_all(engine)


if __name__ == "__main__":
    main()

Error

raceback (most recent call last):
  File "HOMEDIR\AppData\Local\Programs\Python\Python311\Lib\runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "HOMEDIR\AppData\Local\Programs\Python\Python311\Lib\runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher/../..\debugpy\__main__.py", line 39, in <module>
    cli.main()
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 430, in main
    run()
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "HOMEDIR\.vscode\extensions\ms-python.python-2024.2.1\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "Project.git\classes\dummy.py", line 32, in <module>
    main()
  File "Project.git\classes\dummy.py", line 28, in main
    Base.metadata.create_all(engine)
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\schema.py", line 5825, in create_all
    bind._run_ddl_visitor(
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\base.py", line 3254, in _run_ddl_visitor
    conn._run_ddl_visitor(visitorcallable, element, **kwargs)
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\base.py", line 2460, in _run_ddl_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\visitors.py", line 664, in traverse_single
    return meth(obj, **kw)
           ^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 918, in visit_metadata
    self.traverse_single(
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\visitors.py", line 664, in traverse_single
    return meth(obj, **kw)
           ^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 956, in visit_table
    )._invoke_with(self.connection)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 314, in _invoke_with
    return bind.execute(self)
           ^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\base.py", line 1421, in execute
    return meth(
           ^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 180, in _execute_on_connection
    return connection._execute_ddl(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\base.py", line 1529, in _execute_ddl
    compiled = ddl.compile(
               ^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\elements.py", line 307, in compile
    return self._compiler(dialect, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\ddl.py", line 69, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 865, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 910, in process
    return obj._compiler_dispatch(self, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\visitors.py", line 141, in _compiler_dispatch
    return meth(self, **kw)  # type: ignore  # noqa: E501
           ^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 6601, in visit_create_table
    processed = self.process(
                ^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 910, in process
    return obj._compiler_dispatch(self, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\visitors.py", line 141, in _compiler_dispatch
    return meth(self, **kw)  # type: ignore  # noqa: E501
           ^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\compiler.py", line 6632, in visit_create_column
    text = self.get_column_specification(column, first_pk=first_pk)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\dialects\mysql\base.py", line 1818, in get_column_specification        
    column.type._unwrapped_dialect_impl(self.dialect),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 882, in _unwrapped_dialect_impl
    return self.dialect_impl(dialect)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 868, in dialect_impl
    return self._dialect_info(dialect)["impl"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 977, in _dialect_info
    impl = self._gen_dialect_impl(dialect)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 992, in _gen_dialect_impl
    return dialect.type_descriptor(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\engine\default.py", line 589, in type_descriptor
    return type_api.adapt_type(typeobj, self.colspecs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 2323, in adapt_type
    return typeobj.adapt(impltype)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\sql\type_api.py", line 1032, in adapt
    return util.constructor_copy(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\util\langhelpers.py", line 1414, in constructor_copy
    return cls(*args, **kw)
           ^^^^^^^^^^^^^^^^
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\dialects\mysql\types.py", line 185, in __init__
    super().__init__(
  File "Project.git\.venv\Lib\site-packages\sqlalchemy\dialects\mysql\types.py", line 42, in __init__
    raise exc.ArgumentError(
sqlalchemy.exc.ArgumentError: You must specify both precision and scale or omit both altogether.

Additional context

No response

MacBath avatar Mar 10 '24 16:03 MacBath

Hi,

Sorry I guess this issue was missed. I think you will need to use the mysql double directly here. Also note that the setting decimal_return_scale is ignored if you don't set asdecimal=True

@zzzeek Do you think that double should do a better job here while adapting to the driver native type?

CaselIT avatar Mar 16 '24 18:03 CaselIT

https://dev.mysql.com/doc/refman/8.3/en/floating-point-types.html

"FLOAT(M,D)and DOUBLE(M,D) are nonstandard MySQL extensions; and are deprecated. You should expect support for these variants to be removed in a future version of MySQL. "

Since these are deprecated, it's not critical that our own generic DOUBLE doesnt support these. I would say the answer is to not use these deprecated extensions. We can document this caveat

zzzeek avatar Mar 17 '24 14:03 zzzeek

we should likely alter the exception message in mysql.DOUBLE also to reflect this

zzzeek avatar Mar 17 '24 14:03 zzzeek