postgres
postgres copied to clipboard
supabase
Update postgresql.conf to increase max_slot_wal_keep_size
What kind of change does this PR introduce?
Increases max_slot_wal_keep_size default from 1GB
What is the current behavior?
Current behaviour is 1GB by default.
As this is a setting we do not expose, the current default is too low to successfully replicate using a tool like Airbyte (as the WAL will grow too large during the Hourly sync window).
What is the new behavior?
Proposing to set this as 4GB by default but 2GB could be a conservative option.
Additional context
Some relevant issues are noted in the (internal) Notion document
Just curious - have y'all thought about making this configurable per https://supabase.com/docs/guides/platform/custom-postgres-config instead of upping the threshold?
For customers that have a large Supabase instance, it'd be nice to have this number be a % of the total database size as opposed to having it fixed to 1GB (or soon 4GB)
Just curious - have y'all thought about making this configurable per https://supabase.com/docs/guides/platform/custom-postgres-config instead of upping the threshold?
Yes, this should certainly be done. This current setting is to allow replication to function on all instances now but should be configurable in the future (currently it cannot be due to some infra designs but it will be).
Hi :wave: - checking in on the status of this increase.
Our (https://estuary.dev/) users frequently encounter logical replication slot invalidations due to the current 1GB setting. It's far too small in real-world usage, even after users make best-effort mitigations to prevent large transactions.
It would of course be wonderful to see this be configurable, but increasing to 4GB would stop much of the bleeding.
@encima yes, good to merge. Note that this change only affects new projects, and we'll need to roll out the change to existing projects where desired.
