citus icon indicating copy to clipboard operation
citus copied to clipboard

Documentation request: Citus cluster migration

Open matikkane opened this issue 1 year ago • 0 comments

TLDR - I'd like to have documentation covering Citus cluster migration scenarios

Our background (the non-TLDR part) is covered in Citus Slack

To successfully migrate a complete Citus cluster the needed steps are somewhat clear:

  • deploy a new cluster
  • copy the data over and keep it in sync
  • switch over the clients to the new cluster

I believe it's possible to replicate a (current) Citus cluster into empty new Citus cluster (with same number of workers) using PUBLICATION + SUBSCRIPTION across each old and new workers, and across the old and new coordinators. By reading the documentation about CDC the idea is clear - just create pubs+subs not for ALL TABLES, but for the non-metadata user tables. But as we're replicating each worker and coordinators separately it's unclear if citus.enable_change_data_capture brings anything on the table (as we want to each target worker / shard to get 1:1 changes from the source)

It would be awesome if this kind of "migrate your Citus into new hardware" type of scenario would be covered in the documentation.

matikkane avatar Aug 21 '24 12:08 matikkane