powertools-lambda-python
powertools-lambda-python copied to clipboard
Feature request: Idempotent decorator for asynchronous functions
Use case
Would love to be able to use the AsyncBatchProcessor and async_process_partial_response function along with the idempotency utility.
Solution/User Experience
def async_idempotent_function():
def wrapper(func):
@functools.wraps(func)
async def wrapped(*args):
return await func(*args)
return wrapped
return wrapper
Alternative solutions
No response
Acknowledgment
- [X] This feature request meets Powertools for AWS Lambda (Python) Tenets
- [ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET
Thanks for opening your first issue here! We'll come back to you as soon as we can. In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link
hey @esthermcmahon, thank you so much for taking the time to open this feature request. Are you using DynamoDB or Redis as a persistence storage for Idempotency?
The reason we haven't added support for async in idempotency yet is that the AWS SDK is synchronous. This means it'll be blocking and you wouldn't fully benefit from async I/O - e.g., calls to store/retrieve data from Amazon DynamoDB / Redis would block the event loop for a few milliseconds.
for maintainers and anyone looking to help implementing it :)
Off the top of my head, a new async decorator along with these two changes could make it possible (needs to be tested):
- For DynamoDB. We'd need a new
AsyncDynamoDBPersistenceLayerrequiringaioboto3as a dependency. - For Redis. We'd need a new
AsyncRedisCachePersistenceLayer; Redis client supports asyncio so only syntax changes
PS: Adding help wanted to also signal we need to track customer demand. Personally, this is gonna become a common need in Data Engineering and large data pipelines.
Closing this issue as it's been open for a while now, and we didn't have much customer feedback. Please open a new one if it's still an issue.
⚠️COMMENT VISIBILITY WARNING⚠️
This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.