Storage messaging rfc 2.

[arssher]

https://github.com/neondatabase/neon/blob/storage-messaging-2-rfc/docs/rfcs/018-storage-messaging-2.md

[hlinnaka]

With gRPC, is it only used as a message bus, so that if you aren't subscribed to it when a message is sent, it is lost? Or is it possible to connect, and read the current value? IMHO it would be very useful to be able to read the current state, without having to subscribe and listen for updates. That would be very useful when debugging, for example, as you could easily read the current state of a tenant/timeline from one place.

[arssher]

With gRPC, is it only used as a message bus, so that if you aren't subscribed to it when a message is sent, it is lost? Or is it possible to connect, and read the current value? IMHO it would be very useful to be able to read the current state, without having to subscribe and listen for updates. That would be very useful when debugging, for example, as you could easily read the current state of a tenant/timeline from one place.

In the prototype it is only a bus, but it is easy to add some retention period and GET API to fetch the value.

[arssher]

WIth that in mind, I feel like basic websockets are fine?

I wanted to give a shot to it, but realized for that we need to either create a separate connection for each timeline subscription, which sounds bad, or implement multiplexing manually, while grpc gives it us for free. For that, we'd need to

• teach broker that each connection can have multiple subscriptions
• add such protocol message (add sub to this conn, remove it)
• add routing on client side -- e.g. have single task working with socket, while for each sub we'd spawn a user facing object which is able to communicate to this task (get messages from it). I guess tonic has something like that inside.

That said, I'd prefer to stay with gRPC as a first step. Note

• That doesn't increase compilation time, in fact we decrease it compared to current situation.
• We definitely solve the performance issue.
• Infrastructure for running stateless broker built from our code would be created -- so replacing it with something more lightweight would be simple.

CC @SomeoneToIgnore

[arssher]

That said, I'd prefer to stay with gRPC as a first step

So let's proceed with that.