ape
ape copied to clipboard
add asyncio compatibility to the entire ape suite
Overview
Would love to move all suites to be integrated with asyncio. Ape requires a lot of waiting for responses, we could open up the main thread and speed up a lot of what we do for core functionality with the asyncio package. For web2 integrations and for database integrations, this would be a huge addition. Async capable packages are a huge plus.
Specification
synchronous calls:
def some_function(arg):
return do_something(arg)
asynchronous calls:
async def do_something(arg):
return await arg + 5
async def some_function(arg):
return await do_something(arg)
(obviously really rudimentary case here) When combined with requests to APIs, async functions can free up the main thread, and allow for batching of calls, which will decrease the wait time for calls to complete by 10x in many cases. You would fetch for a response, while waiting for that API to receive, then get data, then send that data back, your thread would call the rest of the needed functionality until the await is met.
Dependencies
asyncio - https://docs.python.org/3/library/asyncio.html
For quick guides to asyncio:
- https://www.youtube.com/watch?v=2IW-ZEui4h4
- https://www.youtube.com/watch?v=GpqAQxH1Afc
Deeper Dive:
- https://bbc.github.io/cloudfit-public-docs/asyncio/asyncio-part-1.html
Thought I had: Make it still synchronous out of the box but have async versions of the methods get automatically created.
@generate_async_version
def get_block()
will produce an extra method get_block_async
which returns a co-routine.
This way, it is not a breaking change and noobs can still call the methods easily. Also synchronous and Python feels natural. But then the feature still exists and suites that need async can use those versions and even internally, we can async methods to speed up processing.
@unparalleled-js this is brilliant, and I love the concept. Completely agree on the comment "synchronous and Python feel natural." Python really took async hook, line and sinker following 3.10.5 (might be off on the version, but I think that's the one). And it does some cool stuff for API development. But getting used to development with async is definitely a steep challenge. If we can generate async functions like this, without having to completely tear down Ape, I'm in.
Thought I had: Make it still synchronous out of the box but have async versions of the methods get automatically created.
@generate_async_version def get_block()
will produce an extra method
get_block_async
which returns a co-routine.This way, it is not a breaking change and noobs can still call the methods easily. Also synchronous and Python feels natural. But then the feature still exists and suites that need async can use those versions and even internally, we can async methods to speed up processing.
Can use this and have "ape
as a library" work synchronous out of the box, and then "ape
as an application" use the async API to integrate with click-async
to speed up the overall application flow
https://github.com/click-contrib/asyncclick
See https://linear.app/apeworx/issue/APE-212/spike-add-asyncio-compatibility-to-the-entire-ape-suite for more info
do it!