Worker Nodes Not Recognized as Active in Citus Cluster with Patroni
Problem Description
I'm setting up a PostgreSQL cluster using Citus and Patroni. While I've successfully added worker nodes to the cluster, the system doesn't seem to recognize them as active workers.
Current Behavior
When I query the cluster nodes using SELECT * FROM pg_catalog.pg_dist_node;, I can see all worker nodes and coordinators listed correctly. However, when I run either:
SELECT * FROM citus_check_cluster_node_health();SELECT * FROM citus_get_active_worker_nodes();
Both of these queries return empty results, indicating that the workers aren't being recognized as active.
Expected Behavior
Both queries should return information about the worker nodes that are visible in pg_dist_node.
Environment Information
- Citus version: 13.0-1
- PostgreSQL version: 17.4
- Patroni version: 4.0.4
- OS: Ubuntu 24.04-2
- Deployment method: Proxmox LXC
Steps to Reproduce
- Set up a Citus cluster with Patroni
- Add worker nodes to the cluster
- Run
SELECT * FROM pg_catalog.pg_dist_node;to confirm nodes are added - Run
SELECT * FROM citus_check_cluster_node_health();andSELECT * FROM citus_get_active_worker_nodes(); - Observe that both queries return empty results
Additional Information
We chose not to use Patroni's Citus support because we need to manage multiple databases with Citus. Instead, we created separate clusters: coordinator cluster, worker1 cluster, worker2 cluster, etc. We placed HAProxy+keepalived in front of the system for high availability. We added the coordinator and workers to Citus using their virtual IPs.
Question
What could be causing this discrepancy, and how can I ensure that my worker nodes are properly recognized as active in the cluster?