wrapt Use of ObjectProxy with type()

This is a question (not an issue).

I am using wrapt.ObjectProxy to modify context manager behavior (__enter__, __exit__) of a database connection object (SqlAlchemy Connection). It works as expected with the SqlAlchemy version 1.3.x, but in 1.4.47 it raises an exception in this piece of code (inspection.py, the comments are mine):

https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/inspection.py#L84

    type_ = type(subject) # <--- type() is used, rather than __class__
    for cls in type_.__mro__: # <--- loops over MRO types: (<class 'util.db.ConnectionProxy'>, <class 'ObjectProxy'>, <class 'object'>)
        if cls in _registrars: # <--- not in
            reg = _registrars[cls]
            if reg is True:
                return subject
            ret = reg(subject)
            if ret is not None:
                break
    else:
        reg = ret = None

    if raiseerr and (reg is None or ret is None):
        raise exc.NoInspectionAvailable(  # <--- Throws
            "No inspection system is "
            "available for object of type %s" % type_
        )

Is there a workaround for this? Thank you.

Jun 22 '23 22:06 stolpovsky

Is your code online on GitHub somewhere so can see the rest of the code you have for your derived ObjectProxy? Also how that is used would be helpful as well. It is too hard from just this code snippet to understand what you are trying to do.

Jun 22 '23 22:06 GrahamDumpleton

Not on GitHub, but hopefully this will suffice. If you have a MSSQL database handy, change the database name and the URL and this should be runnable.

import pandas as pd
import urllib
import wrapt
import sqlalchemy


class ConnectionProxy(wrapt.ObjectProxy):

    def __enter__(self):
        try:
            print('enter')
        except LookupError:
            pass
        finally:
            super().__enter__()
            return self

    def __exit__(self, exc_type, exc_value, exc_tb):
        try:
            print('exit')
        except LookupError:
            pass
        finally:
            super().__exit__(exc_type, exc_value, exc_tb)


db_server = {
    'mydb': {
        'prod': 'my.db.host.com',
    },
}

db_conn = {dbname:
    {env: 'Driver={{ODBC Driver 17 for SQL Server}};Server={server};Trusted_Connection=yes;Database={dbname};'.format(
        server=server, dbname=dbname)
        for env, server in m.items()}
    for dbname, m in db_server.items()
}


def get_sqlalchemy_engine(dbname, env):
    url = f'mssql+pyodbc:///?odbc_connect={urllib.parse.quote(db_conn[dbname][env])}'
    engine = sqlalchemy.create_engine(url, fast_executemany=True)
    return engine


def get_sqlalchemy_connection(dbname, env):
    engine = get_sqlalchemy_engine(dbname, env)
    con = engine.connect()
    con.dbname = dbname
    con.env = env

    cp = ConnectionProxy(con)

    return cp


with get_sqlalchemy_connection('mydb', 'prod') as con:
    df = pd.DataFrame({'x': [1, 2, ]})
    df.to_sql('mytable', con, dtype={'x': sqlalchemy.INTEGER})

This is the exception thrown (top of exception stack): File "C:\Python39\lib\site-packages\sqlalchemy\inspection.py", line 71, in inspect raise exc.NoInspectionAvailable( sqlalchemy.exc.NoInspectionAvailable: No inspection system is available for object of type <class '__main__.ConnectionProxy'>

Jun 22 '23 23:06 stolpovsky

You are probably better off monkey patching the connection object instance because sqlalchemy.inspect() derives information from the direct type of an object and doesn't use __class__ to lookup the type of an object, which arguably it should if wants to be compatible with systems that monkey patch stuff. This is the same reason you should use functions like isinstance() rather than comparing type() result to actual types.

Monkey patching __enter__ and __exit__ special methods is complicated though because they are only ever looked up on the type and so you cannot patch them as attributes on the instance.

The following messy code should work though if adapted:

import wrapt

class Connection:
    def __enter__(self):
        print("Connection::__enter__()")
        return self

    def method(self):
        print("Connection::method()")

    def __exit__(self, exc_type, exc_value, exc_tb):
        print("Connection::__exit__()")

print("Test #1")

with Connection() as c:
    c.method()

def get_connection():
    def enter_wrapper(wrapped, instance, args, kwargs):
        try:
            print("enter_wrapper::try")
            return wrapped(*args, **kwargs)
        finally:
            print("enter_wrapper::finally")

    def exit_wrapper(wrapped, instance, args, kwargs):
        try:
            print("exit_wrapper::try")
            return wrapped(*args, **kwargs)
        finally:
            print("exit_wrapper::finally")

    obj = Connection()

    class MonkeyPatchedConnection(type(obj)):
        pass

    wrapt.wrap_function_wrapper(MonkeyPatchedConnection, "__enter__", enter_wrapper)
    wrapt.wrap_function_wrapper(MonkeyPatchedConnection, "__exit__", exit_wrapper)

    obj.__class__ = MonkeyPatchedConnection

    return obj

print("Test #2")

c = get_connection()

with c:
    c.method()

It relies on overriding the __class__ attribute to be a mocked up type for the connection object which provides the context manager special methods.

I don't recollect ever seeing anything ever modify the __class__ attribute before but apparently it can be done. The wrapt module actually blocks trying to update the __class__ attribute via a proxy object, which now I realise there is a use case for updating it, it probably shouldn't block.

Anyway, summary is that SQLAlchemy is not friendly to using proxy objects. It could be argued that it's inspect() method should not use type(subject) but instead use subject.__class__. Although you might suggest such a change to SQLAlchemy, I sort of doubt you would get far since is a rather special corner case and they are likely not to want to upset things even if all regression tests showed a change as still working.