Handling Worker Node Role Changes in Multi-Node Citus/Patorni Setup

[curtisraymond]

I have a multi-node Citus/Patorni deployment in GKE with the following configuration:

• 1 Coordinator node (1 primary + 1 replica)
• 2 Worker nodes (each node has 1 primary + 1 replica)

When a Worker node’s primary pod crashes and restarts under a new IP address, I can successfully use citus_update_node() to update the nodename in the pg_dist_node metadata table. However, if the pod’s role changes from Primary to Replica during this event, I see no corresponding function in Citus that allows me to update the noderole from "primary" to "secondary", hasmetadata from "true" to "false", and metadatasynced from "true" to "false".

I am looking for guidance on how to properly update noderole, hasmetadata, and metadatasynced in pg_dist_node if the role of a pod changes for any reason. While I have noticed that citus_set_node_property() can modify certain pg_dist_node properties, it appears it can currently only change the shouldhaveshards property.

Question: What is the recommended way to update noderole, hasmetadata, and metadatasynced in pg_dist_node to reflect role changes for a Worker node?

Thank you for your assistance!

[ihalatci]

@CyberDem0n any thoughts? From the description feels to me Patroni should reflect the role changes.