tawazi icon indicating copy to clipboard operation
tawazi copied to clipboard

[FEATURE] async support

Open DeoLeung opened this issue 1 year ago • 4 comments

Is your feature request related to a problem? Please describe. dag execute async functions

Describe the solution you'd like could decorate async function the decorator support passing in pool executor in case for running sync

DeoLeung avatar Feb 27 '24 01:02 DeoLeung

Can you please describe the purpose of doing this ? The goal of tawazi is to paralellize the execution of sync code. Are you forced to use async code ?

bashirmindee avatar Mar 14 '24 09:03 bashirmindee

we use fastapi async mode, each node in the execution graph just a http request to other service. using async will be of great help.

DeoLeung avatar Mar 14 '24 11:03 DeoLeung

this feature is not in the roadmap sadly. But if you transform each node to sync, tawazi wil run each node in a separate thread. This would be helpful because your http requests won't block the Python interpreter. This should achieve the results you want.

However I still don't understand why you want to use tawazi in your case... if your code is already completely async, you don't need tawazi... However, if some parts of your code are async and others are sync tawazi is useful for you.

bashirmindee avatar Mar 14 '24 14:03 bashirmindee

we are working on async support here: https://github.com/mindee/tawazi/pull/217

It doesn't support async methods yet but transforms sync functions / ExecNodes into awaitables. Can this be useful to your use case ?

Currently we don't have the incentive to do async support. It could be implemented with ease now though since the code is now adapted to async

bashirmindee avatar Aug 06 '24 14:08 bashirmindee