realtime

realtime copied to clipboard

Hey team,

I originally tried using Cainophile but it hasn't received updates in a couple years. Then I found this project! Nice work.

I see this project as two parts: one part is about subscribing to Postgres and handling log events (a lot of realtime). The other is handling socket connections from clients (realtime_web - Phoenix).

My use case is slightly different, as ideally I'm able to spin up an arbitrary number of processes to subscribe to an arbitrary number of Postgres databases. And then I'll be pushing those events to a destination other than a websocket.

I'd prefer not to fork and further fracture the Elixir community's efforts here :) I'm wondering if we can consolidate our goals. I'm not too familiar with this project yet, but I can imagine taking one of two paths forward:

1. Adding support in this project for subscribing to an arbitrary # of postgres dbs (seems less than ideal, as this knowledge now has to propagate out to the client I'd suppose.)
2. Abstracting out the core subscription/event logic into its own separate package. Then this project builds on top of that, adding the Phoenix/socket/web layer

Again, just some quick thoughts after a brief scan. Let me know what the appetite is. I would implement

Hey @acco - let us discuss this internally and get back to you.

We were already discussing how to move the websockets out, but we also were considering our Workflows engine and how Realtime will work with it. Once we have a clear idea we will drop our notes here.

@kiwicopple Sounds good! Looking forward to your thoughts. Also happy to chat

Hey @kiwicopple -- just checking in to gauge your thoughts here? Cheers

Still disucssing @acco but it's looking likely. Should have some plans for you by next week

@acco sorry for taking so long to get back to ya! I've created issue #146 to track the separation of the "core subscription/event logic" that you mentioned, essentially the Postgres replication functionality, into its own package. Feel free to take a look at that issue and let me know what you think.

We're still discussing internally but I think the direction is that we ultimately want Realtime server to be able to support multiple Postgres databases. I've mentioned this briefly on #146:

Currently, Realtime server can only listen to a single Postgres database but the long-term goal is to support listening to multiple Postgres databases.

@kiwicopple please let me know if this aligns with your own thinking on the matter.

ultimately want Realtime server to be able to support multiple Postgres databases

Yup! Let's make it multi-tenant after we have solved #146

This all sounds great team! Thanks for your responsiveness here. I'll take a look at https://github.com/supabase/realtime/issues/146 shortly

Update here ... we're close to merging in Realtime v2 which supports multiple Postgres servers each as it's own tenant.

This feature is implemented in Realtime V2