magi icon indicating copy to clipboard operation
magi copied to clipboard

`std::sync::RwLock` and tokio runtime

Open borispovod opened this issue 10 months ago • 0 comments

Hey,

I noticed that the std::sync::RwLock being used for the state in the driver might pose some challenges in the future. The RwLock from the standard library is synchronous and has the potential to block the entire runtime, similar to what was observed with the channel in L1.

At the moment, this isn't causing any issues, likely because the batch processing is going step by step. However, as the system evolves, this might lead to complications.

It might be worth considering replacing the standard RwLock with its asynchronous counterpart from tokio. If this route is taken, read/write operations for the state would also need to be made asynchronous. This would entail a significant refactoring effort in derive to use asynchronous constructs, such as replacing Iterators with the Stream.

I'd appreciate hearing your thoughts on this and would welcome any support or guidance if a decision is made to move forward with a PR.

borispovod avatar Sep 07 '23 11:09 borispovod