peerdb icon indicating copy to clipboard operation
peerdb copied to clipboard

MySQL: Need to figure out how to handle binlog retention for large table addition initial loads

Open Amogh-Bharadwaj opened this issue 5 months ago • 1 comments

A MySQL initial load during table addition could take an amount of time greater than the configured value of binlog retention hours on the source MySQL.

In this case, when CDC resumes after the table addition completes, the binlogs we need no longer exist and thus CDC fails, and a resync is the only option.

We need to figure out a way to tackle this phenomenon.

Amogh-Bharadwaj avatar Jul 11 '25 11:07 Amogh-Bharadwaj

Fix would also help postgres avoid slot lag: start reading sync batches right away

serprex avatar Jul 11 '25 12:07 serprex