citus icon indicating copy to clipboard operation
citus copied to clipboard

Worker Nodes Not Recognized as Active in Citus Cluster with Patroni

Open furkangkhsn opened this issue 9 months ago • 0 comments

Problem Description

I'm setting up a PostgreSQL cluster using Citus and Patroni. While I've successfully added worker nodes to the cluster, the system doesn't seem to recognize them as active workers.

Current Behavior

When I query the cluster nodes using SELECT * FROM pg_catalog.pg_dist_node;, I can see all worker nodes and coordinators listed correctly. However, when I run either:

  • SELECT * FROM citus_check_cluster_node_health();
  • SELECT * FROM citus_get_active_worker_nodes();

Both of these queries return empty results, indicating that the workers aren't being recognized as active.

Expected Behavior

Both queries should return information about the worker nodes that are visible in pg_dist_node.

Environment Information

  • Citus version: 13.0-1
  • PostgreSQL version: 17.4
  • Patroni version: 4.0.4
  • OS: Ubuntu 24.04-2
  • Deployment method: Proxmox LXC

Steps to Reproduce

  1. Set up a Citus cluster with Patroni
  2. Add worker nodes to the cluster
  3. Run SELECT * FROM pg_catalog.pg_dist_node; to confirm nodes are added
  4. Run SELECT * FROM citus_check_cluster_node_health(); and SELECT * FROM citus_get_active_worker_nodes();
  5. Observe that both queries return empty results

Additional Information

We chose not to use Patroni's Citus support because we need to manage multiple databases with Citus. Instead, we created separate clusters: coordinator cluster, worker1 cluster, worker2 cluster, etc. We placed HAProxy+keepalived in front of the system for high availability. We added the coordinator and workers to Citus using their virtual IPs.

Question

What could be causing this discrepancy, and how can I ensure that my worker nodes are properly recognized as active in the cluster?

furkangkhsn avatar Mar 17 '25 10:03 furkangkhsn