fluvio
fluvio copied to clipboard
tech debt: refactor `notify_followers` from `write_record_set`
In https://github.com/infinyon/fluvio/blob/master/crates/fluvio-spu/src/replication/leader/replica_state.rs#L338, both notify_followers are not strictly not necessary since they could be implemented as offset listener.
I'll be taking a stab at this!
So after doing some digging I'm wondering what the best way to go about this is without adding cognitive and technical overhead.
It seems the intent of this ticket is to achieve the following:
- Select on a new listener in
FollowerHandler's event_loop. - Rather than call notify_followers, we'll emit an event.
- The event is captured in the event loop mentioned in step 1, and we proceed to do what
notify_followersis essentially doing which is publish updates, which is then captured in this part of the event loop.
Although this could work, I don't think it's more advantageous than the current approach, which is publishing updates for followers immediately after the leader writes new records. The advantage of coupling notify_followers with write_record_set is that we know, then and there, which leader just got updated and which followers subsequently need their offsets updated.
By decoupling write_record_set with notify_followers and having the latter be event-driven we'd have to later deduce which followers need updates published in the event loop.
Please let me know if I'm off the mark! Happy to take suggestions. I'll continue to sit on various approaches.
Also, this test is what I'm using to verify correctness.