sled icon indicating copy to clipboard operation
sled copied to clipboard

create watch_prefix_sink() (was: Impl Stream for Subscriber)

Open rrichardson opened this issue 3 years ago • 1 comments

Use Case: Use Subscriber as if it were a Stream of Events instead of a Future of a single event multiple times.

As the documentation for Future and Stream states:

 The Stream trait is similar to Future but can yield multiple values before completing, similar to the Iterator trait from the standard library:

It seems to me that the Future should actually be a Stream, since Subscriber is intended to return some unbounded stream of Events instead of a single event.

Proposed Change: Implement Stream for Subscriber and possibly remove the Future implementation.

Actually, looking at the impl very little code would actually change. I don't entirely understand the motivation for a channel of oneshot channels, but I'm sure there is a very good reason. ( If I had to make a wild-ass guess, I'd say that the inner one-shots are results of not-yet-completed transactions, and, in some cases, the transaction may be aborted, so it doesn't actually contain data)

Who Benefits From The Change(s)? Me. Also new users who are trying to understand how to subscribe to a stream of events.

Alternative Approaches

Keep the multi-use feature, and document it accordingly?

rrichardson avatar Mar 29 '21 19:03 rrichardson

Actually. Having worked through the Broadcast -> Subscriber process in sled, and then having to morph the subscriber to fit the needs of the stuff I'm building...

I have a new opinion:

watch_prefix should just take a Sink<Item=Event> + Clone (like Receiver<Event>)

At every write/commit, ReservedBrodacast::complete can just iterate through its list of Sinks and send(event.clone()) on each one. That's it. No mucking about with futures and polling and OneShots.. Let the user worry about the plumbing and ensuring they have subscribers, etc.

This would make things more efficient for use-cases like mine, where I am streaming output to any number of clients, and that output is comprised of any number of watches on various prefixes. So whatever muxing and demuxing is done by Sled, I have to turn around and re mux and demux it. Instead, I can just pass my own specialized Sink to watch_prefix and new writes could be sent straight to my service's stream-output.

rrichardson avatar Apr 03 '21 02:04 rrichardson