Hey,
I noticed that the std::sync::RwLock being used for the state in the driver might pose some challenges in the future. The RwLock from the standard library is synchronous and has the potential to block the entire runtime, similar to what was observed with the channel in L1.
At the moment, this isn't causing any issues, likely because the batch processing is going step by step. However, as the system evolves, this might lead to complications.
It might be worth considering replacing the standard RwLock with its asynchronous counterpart from tokio. If this route is taken, read/write operations for the state would also need to be made asynchronous. This would entail a significant refactoring effort in derive to use asynchronous constructs, such as replacing Iterators with the Stream.
I'd appreciate hearing your thoughts on this and would welcome any support or guidance if a decision is made to move forward with a PR.
Hey,
I noticed that the
std::sync::RwLockbeing used for the state in the driver might pose some challenges in the future. TheRwLockfrom the standard library is synchronous and has the potential to block the entire runtime, similar to what was observed with the channel in L1.At the moment, this isn't causing any issues, likely because the batch processing is going step by step. However, as the system evolves, this might lead to complications.
It might be worth considering replacing the standard
RwLockwith its asynchronous counterpart fromtokio. If this route is taken, read/write operations for the state would also need to be made asynchronous. This would entail a significant refactoring effort in derive to use asynchronous constructs, such as replacing Iterators with the Stream.I'd appreciate hearing your thoughts on this and would welcome any support or guidance if a decision is made to move forward with a PR.