neon
neon copied to clipboard
neondatabase
Move WALProposer into an extension
We have a lot of Neon-specific code related to walproposer in src/backend/replication. Let's move that to an extension, under contrib/neon. See https://github.com/neondatabase/neon/blob/main/docs/core_changes.md#walproposer. This takes us one step closer to running vanilla Postgres on Neon storage.
This probably requires adding a new API or hook in core Postgres. That's OK. We'll then need to then submit that API to upstream, but that's a separate task.
https://github.com/neondatabase/postgres/pull/96 was an earlier attempt at this.
Several weeks ago we talked a bit about this with @arssher and reached a conclusion that we can remove all Neon-specific code from walsender and instead interact with replication slots directly from walproposer. Currently, we are bypassing most of the walsender code with our patches and it makes sense to remove all our patches from walsender and then just copy some essential code from walsender to our new extension. It seems like replication slots API already have everything we need for walproposer to work, so probably we don't need to add any new hooks there. WDYT?
Done in commit f99ccb5041! Opened https://github.com/neondatabase/neon/issues/2307 for the next step.
Several weeks ago we talked a bit about this with @arssher and reached a conclusion that we can remove all Neon-specific code from walsender and instead interact with replication slots directly from walproposer. Currently, we are bypassing most of the walsender code with our patches and it makes sense to remove all our patches from walsender and then just copy some essential code from walsender to our new extension. It seems like replication slots API already have everything we need for walproposer to work, so probably we don't need to add any new hooks there. WDYT?
Commit f99ccb5041 removed most of the code changes from src/backend/replication. Do you think we could get rid of the rest, too? If so, sounds great!
Do you think we could get rid of the rest, too?
Not sure, I think f99ccb5041 implemented most of the things I was talking about. I still think it's possible to clean up contrib/neon/walproposer_utils.c more as I wrote in https://github.com/neondatabase/postgres/pull/188#discussion_r938942341, but I won't insist on that.