sqlakeyset
sqlakeyset copied to clipboard
djrobstep
Support SQLAlchemy 2.0-style ORM queries
perform_paging is broken when running SQLAlchemy 1.4.7.
This fix worked for me:
def perform_paging(q, per_page, place, backwards, orm=True, s=None):
if orm:
selectable = orm_to_selectable(q)
s = q.session
column_descriptions = q.column_descriptions
keys = orm_query_keys(q)
else:
selectable = q
column_descriptions = q._raw_columns
to
def perform_paging(q, per_page, place, backwards, orm=True, s=None):
column_descriptions = q.column_descriptions
if orm:
selectable = orm_to_selectable(q)
s = q.session
keys = orm_query_keys(q)
else:
selectable = q
Can you provide the backtrace for the error you encountered, and some information about the query that caused it? I just ran our test suite with sqlalchemy 1.4.7 and everything passed.
def test_pagination(db: Session) -> None:
from sqlakeyset import select_page
from sqlalchemy import select
select_page(db, select(Item).order_by(Item.id.desc()), 2)
db = <sqlalchemy.orm.session.Session object at 0xffffa2ed6eb0>
def test_pagination(db: Session) -> None:
from sqlakeyset import select_page
from sqlalchemy import select
> select_page(db, select(Item).order_by(Item.id.desc()), 2)
app/tests/graphql/test_hello.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.9/site-packages/sqlakeyset/paging.py:250: in select_page
return core_get_page(s, selectable, per_page, place, backwards)
/usr/local/lib/python3.9/site-packages/sqlakeyset/paging.py:164: in core_get_page
paging_result = perform_paging(
/usr/local/lib/python3.9/site-packages/sqlakeyset/paging.py:86: in perform_paging
mapped_ocols = [find_order_key(ocol, column_descriptions) for ocol in order_cols]
/usr/local/lib/python3.9/site-packages/sqlakeyset/paging.py:86: in <listcomp>
mapped_ocols = [find_order_key(ocol, column_descriptions) for ocol in order_cols]
/usr/local/lib/python3.9/site-packages/sqlakeyset/columns.py:415: in find_order_key
ok = derive_order_key(ocol, desc, index)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ocol = <OC: item.id DESC>
desc = Table('item', MetaData(), Column('id', Integer(), table=<item>, primary_key=True, nullable=False), Column('title', Str...eignKey('location.id'), table=<item>), Column('owner_id', Integer(), ForeignKey('user.id'), table=<item>), schema=None)
index = 0
def derive_order_key(ocol, desc, index):
"""Attempt to derive the value of `ocol` from a query column.
:param ocol: The :class:`OC` to look up.
:param desc: Either a column description as in
:attr:`sqlalchemy.orm.query.Query.column_descriptions`, or a
:class:`sqlalchemy.sql.expression.ColumnElement`.
:returns: Either a :class:`MappedOrderColumn` or `None`."""
if isinstance(desc, ColumnElement):
if desc.compare(ocol.comparable_value):
return DirectColumn(ocol, index)
else:
return None
> entity = desc["entity"]
E TypeError: 'AnnotatedTable' object is not subscriptable
/usr/local/lib/python3.9/site-packages/sqlakeyset/columns.py:356: TypeError
I think I figured it out.
If you change
def test_core2(dburl):
with S(dburl, echo=ECHO) as s:
sel = select([Book.score]).order_by(Book.id)
check_paging_core(sel, s)
to
def test_core2(dburl):
with S(dburl, echo=ECHO) as s:
sel = select(Book).order_by(Book.id)
check_paging_core(sel, s)
it fails
Have just been through this. It will take significant refactoring to support async/await because there are two or three I/O places in the internals. The linked gist is what I now have in my codebase to make 2.0 style with an AsyncSession work.
We ended up forking. As I understand it, SQLAlchemy 2.0 will have deprecated session.query and allowed declarative models as selectables instead - so all queries are in the core style, then sent to be session.execute-ed. We are also using the asyncio extension so we have to await all those. I don't know if it's worth trying to support everything right now, but if anyone has our requirements we are happy to share our fork.
For reference we are using this library to support Relay-style pagination for Apollo GraphQL clients.
@jokull sharing a fork might be useful, I suppose. My team is also trying to implement cursor pagination based on async sqlalchemy.
Currently thinking about making a fork myself, would love to see your implementation)
Sorry for my silence here, I just don't have much time to be spending on open source work at the moment, and I assume djrobstep is the same. We're very happy to accept PRs so long as backwards-compatibility is not broken - if our test suite for 1.3/1.4 still passes and it adds 2.0 support, it'll get merged.
@confar Here's the fork me and @jokull ended up writing. The fork uses the 2.0 style exclusevily and uses the asyncio integration of SQLAlchemy.
My impression is that async variants of Python projects tend to become their own repos, like aiohttp or aioredis. So I'd say it's hard to have both 1.3/1.4 and 2.0-async syntax under the same library, but perhaps possible.
