wrapt

wrapt copied to clipboard

I'm trying to use wrapt.ObjectProxy to wrap an aiohttp response. I try to use the proxy in an async with statement, but it errors out with AttributeError: __aexit__

In [1]: class TestContextManager:
    async def __aenter__(self):
        pass
    async def __aexit__(self, exc, exc_type, tb):
        pass
   ...:

In [2]: import wrapt

In [3]: proxy = wrapt.ObjectProxy(TestContextManager())

In [4]: async def test():
    async with proxy:
        pass
   ...:

In [5]: import asyncio

In [6]: asyncio.get_event_loop().run_until_complete(test())
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-6-15c8284d456a> in <module>()
----> 1 asyncio.get_event_loop().run_until_complete(test())

/Users/graeme/.pyenv/versions/3.5.1/lib/python3.5/asyncio/base_events.py in run_until_complete(self, future)
    335             raise RuntimeError('Event loop stopped before Future completed.')
    336
--> 337         return future.result()
    338
    339     def stop(self):

/Users/graeme/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py in result(self)
    272             self._tb_logger = None
    273         if self._exception is not None:
--> 274             raise self._exception
    275         return self._result
    276

/Users/graeme/.pyenv/versions/3.5.1/lib/python3.5/asyncio/tasks.py in _step(***failed resolving arguments***)
    237                 # We use the `send` method directly, because coroutines
    238                 # don't have `__iter__` and `__next__` methods.
--> 239                 result = coro.send(None)
    240             else:
    241                 result = coro.throw(exc)

<ipython-input-4-04eb2b4a5385> in test()
      1 async def test():
----> 2     async with proxy:
      3         pass
      4

AttributeError: __aexit__

I am curious as to why you are wrapping the context manager in the first place. Not suggesting you are doing anything wrong, I just haven't ever sat down to understand how all these new keywords work.

Your test case works if doing something like:

class TestContextManager:
    async def __aenter__(self):
        pass
    async def __aexit__(self, exc, exc_type, tb):
        pass

import wrapt

class ObjectProxy(wrapt.ObjectProxy):
    def __aenter__(self):
        return self.__wrapped__.__aenter__()

    def __aexit__(self, *args, **kwargs):
        return self.__wrapped__.__aexit__(*args, **kwargs)

#proxy = wrapt.ObjectProxy(TestContextManager())
proxy = ObjectProxy(TestContextManager())

async def test():
    async with proxy:
        pass

import asyncio

asyncio.get_event_loop().run_until_complete(test())

but I am not sure if it is enough for me to do similar thing in wrapt in Python and C variants of wrapper, or whether at C level I need to do anything with slots on the C object.

There is also the __await__() method. Should it get the same treatment, or is there something special about that and will problems arise if it exists on the proxy but it didn't exist on what was wrapped?

Any extra information you can give me to help enlighten me would be most appreciated.

I encountered this when using ObjectProxy to wrap the result of aiohttp.request. It returns an object that can be used as an async context manager, and also a coroutine. I want to change it's behaviour a tiny bit, and I thought an ObjectProxy would be a good way to do that.

I have worked around this issue by defining __aenter__ & __aexit__ methods manually on my object proxy, similar to your code sample above.

I think (though I'm not an expert) defining functions that just return the result of the async methods on the wrapped object is fine - async functions and normal functions are called the same way, it's just that async functions return an awaitable, and their actual code doesn't run until that is awaited on.

__aiter__ and __anext__ are similar, though it does appear that python specifically checks for __aiter__ methods when used in an async for:

In [1]: class Test():
   ...:     pass
   ...:

In [2]: async def test():
   ...:     async for a in Test():
   ...:         pass

In [8]: asyncio.get_event_loop().run_until_complete(test())

TypeError: 'async for' requires an object with __aiter__ method, got Test

So I guess providing these on the wrapper when they didn't exist on the actual object itself could change the behaviour slightly...

I think __await__ is a different case - it's not actually an async function, but a normal function that returns an iterator. Again, it does seem like python does some special checks on objects that are used in await expressions.

In [9]: class Test:
   ...:     def __await__(self):
   ...:         pass
   ...:

In [10]: async def test():
   ....:     await Test()
   ....:

In [11]: asyncio.get_event_loop().run_until_complete(test())

TypeError: __await__() returned non-iterator of type 'NoneType'

In [14]: class Test():
   ....:     pass
   ....:

In [15]: asyncio.get_event_loop().run_until_complete(test())

TypeError: object Test can't be used in 'await' expression

Having proxy methods for __aiter__ and __anext__ should not be an issue because you would have to be using async for for that test to happen, in which case they would need to exist anyway. At all other times I can't see that it would matter if they exist on the wrapper but the wrapped object doesn't provide them. Same issue arises with __iter__ and __next__, their existence only matters when the code you write does something that requires them. Yes?

Yeah, that seems reasonable.

I suppose the only case where it'd make a difference is someone might mis-use an object proxy in an async for and get a slightly different exception from normal, but that's definitely an extreme edge case.

FYI the unittest mocking framework does special things for __enter__ and __exit__ (MagicMock), similarly the asynctest module owner recently fixed wrapping for __aenter__ and __aexit__: https://github.com/Martiusweb/asynctest/issues/29. So perhaps there needs to be a MagicObjectProxy or something

ping