Dealing with async functions

[keis]

When there is a async function we want to test

async def frobnicate(arg0):
    raise ValueError("Could not do it")

naïvely we may write (spoiler alert this will not work)

assert_that(calling(frobnicate).with_args(5), raises(ValueError))

As a first suggestion it would be nice if the error message could guide me in the right direction like

Expected: Expected a callable raising <class 'ValueError'>
     but: Coroutine returned (Did you mean to await the call)

There's a couple of ways of testing this function that does work but they look a bit clunky or have other issues. Does anyone have suggestions on how to better to do this?

a) using run_until_complete

def calling_async(callable: Callable, *, loop: Loop):
    def wrapper(*args, **kwargs):
        loop.run_until_complete(callable(*args, **kwargs))
    return calling(wrapper)

assert_that(calling_async(frobnicate, loop=loop).with_args(5), raises(ValueError)

Looks pretty neat but has the big down side it's not possible to use when the test method itself is async (like when using pytest-async)

b) testing the result function of a future instead

async def resolved(obj):
    fut = asyncio.ensure_future(obj)
    await asyncio.wait([fut])
    return fut

async def test_frobnicate():
    future = await resolved(frobnicate(5))
    assert_that(calling(future.result), rasise(ValueError))

This is arguably the more sane approach but with the assert line only being calling(future.result) it loses some context.

[brunns]

I've not done any async work myself, so I can't make any suggestions right now. Do we think there might be a need for some async specific matchers? What might they look like, from a test's point of view?

[keis]

I need to explore this a bit more myself but I wanted to get some ideas from what others might be doing already.

Having async specific (or rather future specific) matchers could makes sense based a bit on the b option above. Trying to make hamcrest deal with async methods and running a asyncio loop is just a can of worms not worth opening. Consider having one or two matchers that works on Future[T] and perhaps paired with a helper function like resolved.

I imagine it could look something like

assert_that(await resolved(frobnicate(5)), future_with_exception(ValueError))
assert_that(await resolved(frobnicate(6)), future_with_result(equal_to(13)))

Wiich would make mypy/pyright able to reason about this calls a bit.

The biggest immediate gain would be from introducing some warning or specific error message when raises run into a coroutine, I can prepare a PR to that effect if you think that makes sense.

[offbyone]

what you're describing makes sense to me, at least in principle. I would want to see examples in the pull request, but I can easily see merging it.

if possible make sure that there is clear usage demonstrated in the PR.