pglogical icon indicating copy to clipboard operation
pglogical copied to clipboard

Does table replication with row_filter require more memory?

Open kostyay opened this issue 1 year ago • 3 comments

Hi I'm trying to replicate a table with a row_filter defined (start_time > current_date - interval '4 months') and it causes memory usage to spike to 100% which ultimately causes the replication to go down as the process crashed with OOM. Its a table with about 5 mil rows on a 6 vcpu 16 gig Postgres 14 machine. If I try to sync the entire table without row_filter it finishes successfully. I tried creating an index on start_time but it didn't help. Is this the expected behaviour? Is there anything I can do to resolve this (besides adding more memory)? Thanks!

kostyay avatar May 29 '24 22:05 kostyay

We also experience this issue, in a similar set of circumstances - large table, which we're trying to replicate using a row filter, on an r7g.4xl postgres RDS, and the process brings the whole instance down - the main process gets OOM killed. Without a row filter however, the replication is fine.

Did you find any ways to optimise this, @kostyay?

danrough avatar Jul 22 '24 07:07 danrough

Unfortunately not

kostyay avatar Sep 04 '24 19:09 kostyay

We moved to using the native replication with a row filter, rather than using the pglogical module, and we have been able to progress with this. So we've now abandoned the use of the pglogical module for anything other than replicating sequences.

danrough avatar Sep 05 '24 09:09 danrough